Helldivers 2 API

[dealloc]

Hello there!

Let me preface this by saying thank you to the community who has been contributing to this API so far, both with suggestions, endpoints, bug reports and PR's.
The aim of this API is to become one central go-to point for everyone looking to build upon Helldivers' 2 information.

While the API is currently designed in Elixir, and I love me some good Elixir, I feel like there's several reasons I'd like to migrate away to a new language:

Open for contributions

Much as I love Elixir, it's not exactly the most widely known language out there, which makes the barrier for people looking to contribute to the API just that bit higher, ideally we'd migrate to a more widely known language

Immutable data structures

Elixir is a language that uses immutable data structures, modifying something makes a copy of it and everything referencing the original object keeps the original data.
This makes working with circular references quite difficult (I'm looking at you planet waypoints 👀 )
A language with mutable objects (to a certain degree at least) would make this sort of parsing a lot easier

API design

While there's definitely already tooling in place in the current version for OpenAPI etc, it's quite painful to maintain multiple schemas for the OpenAPI and the domain logic schema's itself, whereas languages like C# allow generating this directly from the domain logic

Reusable data

Currently most re-usable data can be found in the priv folder for things like planets, sectors etc.
Ideally we'd find a way so that this information can be shared to all other projects in a language agnostic way, perhaps in it's own repository that could be updated centrally

Experience

Although I have some professional experience working in Elixir, I'm a C# developer in my day 9-to-5 job, and I can confidently say I'm much more proficient in security, design, resource management and performance in languages like C# than I am in Elixir.

Conclusion

I'm opening this discussion to give a voice to the people who are currently using the API.
In short, I'm open to rewriting the entire API from scratch in a new language (current frontrunner is C#, though I am not opposed to other suggestions), and with it most likely some serious changes to the API design.
The schema as it's currently designed (closely resembling the underlying API, with some syntax sugar on top) would probably remain existing to 1. avoid breaking changes and 2. allow people who are directly using the API to transition more smoothly
but I'd like to create some more high level endpoints for common use cases in websites being built by the Helldivers 2 community.
Another idea I'm playing with is GraphQL, which would allow for much more efficient data fetching for websites and websockets to allow people to "subscribe" to information rather than having to rely on a polling mechanism as most API's currently have to.

So this is your chance to provide input on how you use this API, or what you'd like to see from this API and hopefully we can create the one API to rule them all.

Cheers!
~ @dealloc

[lambstream]

First of all, thank you for laying the ground work! It's daunting to think about starting an API when there's nothing to go off of.

I think GraphQL could be interesting to explore because as you mentioned it would be a lot more efficient.

Personally C# is fine for me but I'd love to hear others thoughts on what we could use as an alternative.

Lastly moving the static data to a seperate CDN project or something similar sounds like a good idea.

+1

[dealloc]

thank you for the kind words @lambstream
I'm currently exploring the possibilities with .NET and AOT, C# does seem like the language getting most votes (both here and in the Discord), so hopefully that would lower the barrier for new contributions

[CallumHoughton18]

+1 for C# .NET too :D

[milogert]

Well I love Elixir, so I'm biased 😂

Having the self-healing GenServer as well as :ets keeps everything in a single technology, which I think is great (i.e. no needing to learn/setup/maintain K8s or Redis).

So my vote is officially for Elixir ✅

Edit: also, thank you for starting this. I love to see Elixir in the wild and so far this project has been great to work with.

[dealloc]

I'm obviously a big fan of Elixir as well (which is why it's currently written in it), but it's admittedly not a language the wider community will be able to contribute in.
Aside from immutable data structures being a pain with circular references in Elixir, we don't make heavy use of OTP at the moment, though I'll definitely admit I'd miss :ets if I'd rewrite in another language

[milogert]

It's there a big need for recursive data structures here? Especially with returning plain JSON from the endpoints? I might have a total misunderstanding of how to use the data returned form the endpoints so please correct me if I'm wrong!

If you were looking for something like pseudo nesting Ecto does that with preloading foreign keys from databases. So when you ask for like planets you could also say "and give me the next waypoints" N levels deep or something.

Just spitballing here 🤷‍♂️

[dealloc]

The nested data structures aren't necessarily a part of the API, but they make modelling internal data structures a lot easier.
The primary reason for this switch however remains to reach a larger audience with a more 'common' language

[dbrennand]

I really like the approach of this project and providing an open source community API. Personally, I'd veer towards Python due to its popularity (contribution barrier to entry). Packages such as FastAPI (based on OpenAPI spec) and strawberry could be used to provide the API and GraphQL functionality respectively. FastAPI has auto generating docs etc.

[dealloc]

hi @dbrennand we're currently going for C#, primarily because the main contributors have most experience with this and it's a fairly widely used language (which was part of why this discussion was opened in the first place).
However, if you love Python for an API like this you should check out helldivers-2/diveharder_api.py