Spark API change proposal

[khalwat]

I have a proposal to change the Spark API, predicated on the idea that the real purpose of Spark is:

• To allow the rendering of Twig templates
• To allow the running of various Craft actions like saving a User
• A way to pass state back and forth from the frontend to Twig templates, and vice versus

I think rather than creating a new "sort of similar but different" API on top of Spark isn't the best route, because:

• It forces you to learn a new thing you will only use with Craft & Spark
• It obfuscates the Datastar API that you will end up needing to use for all but the simplest of things
• You aren't able to leverage the Datastar documentation, example code and support as easily, rather you'll have to create your own

I realize the desire to decouple Spark from Datastar, but the reality is that unless you find another library that uses the data- attributes in exactly the way Datastar does, your code is going to end up being tightly coupled to Datastar anyway.

So instead, embrace it.

My proposal largely centers around embracing the existing APIs in Datastar, Twig, and Craft and using them rather than creating your own "new and similar but a little different" API on top.

It also has the thread running through it that we shouldn't obfuscate what's actually going on, because that generally leads to a steeper learning curve in the long run.

Here's what I propose changing:

Eliminating action verbs

• data-on-click="{{ spark.get('some/template.twig') }}" -> data-on-click="$$get({{ spark.renderTemplate('some/template.twig') }})"
• data-on-click="{{ spark.put('some/template.twig') }}" -> data-on-click="$$put({{ spark.renderTemplate('some/template.twig') }})"
• data-on-click="{{ spark.post('some/template.twig') }}" -> data-on-click="$$post({{ spark.renderTemplate('some/template.twig') }})"
• data-on-click="{{ spark.patch('some/template.twig') }}" -> data-on-click="$$patch({{ spark.renderTemplate('some/template.twig') }})"
• data-on-click="{{ spark.delete('some/template.twig') }}" -> data-on-click="$$delete({{ spark.renderTemplate('some/template.twig') }})"

In other words, have Spark just render the appropriate URL that will render a given Twig template, rather than output the Datastore JavaScript code surrounding it, similar to how Craft's baked in {{ actionUrl() }} works.

This makes it more transparent in terms of what is actually going on (we're rendering a Twig template), and allows you to use existing Datastor sample code by just changing the URL.

Alternatively, you could use the verb .renderFragment() if you wanted to distinguish the templates from regular old Twig templates.

This becomes more important when you realize that everything that is in the data-on-* event handlers is just JavaScript code, and can be (and often will be) more elaborate than just rendering an endpoint, as shown in this example: https://datastar.fly.dev/examples/dialogs_browser

<button
  id="dialogs"
  data-store="{prompt:'foo',confirm:false}"
  data-fetch-url=""
  data-on-click="$prompt=prompt('Enter a string',$prompt);$confirm=confirm('Are you sure?');$confirm && $$get('/examples/dialogs___browser/sure')"
>
  Click Me
</button>

Used with Spark and my proposed changes, this would just become:

<button
  id="dialogs"
  data-store="{prompt:'foo',confirm:false}"
  data-fetch-url=""
  data-on-click="$prompt=prompt('Enter a string',$prompt);$confirm=confirm('Are you sure?');$confirm && $$get({{ spark.renderTemplate('/examples/dialogs___browser/sure.twig') }})"
>
  Click Me
</button>

I think this makes it clearer what is actually going on, and lets you leverage existing Datastar documentation and example code easier.

Namespacing the store

The other thing I'd like to see is that since you reference items in the Datastore with a $ in front of them in JavaScript on the frontend (e.g., accessing the search item in the store is done via $search), I'd ideally like to see the store variables that are injected into the Twig template be prefixed with $ too, to distinguish them from regular Twig variables.

Sadly, there doesn't appear to be a way to do this, because $search, for example, is an invalid variable name in Twig.

What you could do, though, is put all of the variables in one object, rather than polluting the template context with a potentially huge number of variables that could collide with variables you use in your templates with transparency.

So instead of the search variable being set in the Twig context due to the fact that search is in your Datastore context, you'd reference it via store.search or some other variable name of your choosing.

I think this makes it more explicit in terms of what's happening in your templates (you can plainly see that the value is coming from the Datastar store rather than it just being a magically set variable) and it prevents namespace collisions that will likely happen inadvertently for templates of any complexity. This is especially important if different people are working on different aspects of the code.

Automatic updating of the store

The other thing that namespacing the store would let you do pretty cleanly would be you could eliminate entirely the {% do spark.setStore({ search: 'foo' }) %}.

Instead, treat the store. as a special reactive object store that automatically does a diff on any changes in it, and sends them back to the Datastar frontend.

Granted, Twig makes this slightly annoying, in that you have to use the | merge filter to update the store object, but it's a pattern people are used to and familiar with working with in Twig.

The name

I'd also just drop the pretense of Spark being its own new, differently branded thing, and call it Craft Datastar

The Dual APIs

Spark has two APIs:

• Frontend -> Backend - The API used in regular Twig templates that (mostly) renders JavaScript to help generate Datastar expressions that communicate with the Craft backend (hereinafter referred to the "Frontend API")
• Backend -> Frontend - The API used in fragment Twig templates that renders messages in the form of Server Side Events that communicate with Datastar on the frontend (hereinafter referred to as the "Backend API")

Currently, both APIs are available under the spark. namespace, but there's a problem: the Frontend APIs can only be used in the regular Twig templates, and the Backend APIs can only be used in the fragment templates.

Also currently, it's a little strange in that what's actually happening is obfuscated. Fragment templates have an implicit Fragment SSE that sends along whatever the Twig template renders. But also there are methods like spark.delete() and spark.redirect() and such that send additional explicit events.

I think it is confusing that parts of the spark. API work in regular Twig templates but don't work in fragment templates, and other parts of the spark. API work in fragment templates, but don't work in regular Twig templates.

My general advice when writing an API is to simplify but don't obfuscate. People more easily get their head around APIs that are consistent in terms of scope and functionality.

So much as I've advocated for Spark to spend less work making it's own API with the various spark.get/spark.put verbs, and instead just have a spark.renderTemplate method, I'm also going to suggest making the fragment templates declaratively show what's going on, rather than hide it.

So with that in mind, instead of a bifurcated API where parts works in regular Twig templates, and part works in fragment templates, here's what I'm proposing:

• The spark. API is available only in regular Twig templates
• In fragment templates, we get rid of the implicit fragment SSE, and instead do it explicitly
• All events then work the same way

So what I propose is explicitly showing people that they aren't really rendering a Twig template, they are sending events back to Datastar on the frontend:

{% send fragment %}
    {# In here is the HTML/Twig code that renders whatever fragment is being sent back as a SSE #}
{% endsend %}

And we do the same thing for the other events:

{% send console log %}
    {# In here is the HTML/Twig code that renders whatever should be logged to the console as a SSE #}
{% endsend %}

{% send redirect %}
    {# In here is the HTML/Twig code that renders whatever redirect URL should be sent back as a SSE #}
{% endsend %}

{% send delete %}
    {# In here is the HTML/Twig code that renders whatever element `id` should be deleted as a SSE #}
{% endsend %}

(you can add whatever modifiers you need to for the send <event>, e.g. send console **log** as it makes sense)

Most of the time people will just be doing {% send fragment %} which isn't that far off what you're doing already with your {% fragment %} tags currently, it's just more explicit. And then for the more rare times people are sending back the other events, you have a consistent way to do it.

Now we have a consistent way to represent all of these things, and we're doing it in a way that makes it explicit what is actually happening: we're sending events back to Datastar on the frontend.

And we also now have separated the regular spark. API that's used in regular Twig templates from the API that is used in fragment Twig templates clearly and cleanly.

This way is also more easily extensible when you need to add more event types that Spark can send back as Datastar evolves.

To be nice, trim the whitespace from the beginning and end of everything in the {% send %} blocks.

If you don't like the send verb, you could also do event or sendEvent instead, but I think send is pithy and conveys what is actually going on better.

[bencroker]

Thanks for the proposal!!

Eliminating action verbs

I considered doing this initially, as I agree that it keeps you closer to the “metal”. What swung me in the direction I took was:

1. I find the $$get('...') syntax kind of ugly and data-on-click="$$get('{{ spark.renderTemplate('...') }}') rather verbose, and thought I could eliminate some of the overhead of having to understand what $$ is by using the cleaner, terser syntax data-on-click="{{ spark.get('...') }}". Note also the missing single apostrophes in your proposal, which further demonstrates the complex syntax that it would result in.
   Your proposal:
   data-on-click="$$get({{ spark.renderTemplate('some/template.twig') }})"
   Which should read:
   data-on-click="$$get('{{ spark.renderTemplate('some/template.twig') }}')"

2. For all non-GET requests, Spark sends a CSRF token as a query parameter. If the API was data-on-click="$$post('{{ spark.renderTemplate('...') }}')" then Spark would not know what kind of request this was and would therefore be unable to know whether it should send a CSRF token or not. It could, of course, always include a CSRF token, but that seems wasteful for GET requests since the length of submitted URIs is not unlimited.

3. I agree that this doesn’t read great: data-on-click="$clicked = 1; {{ spark.post('some/template.twig') }}". I could add a spark.url() function in addition to what currently exists, so that data-on-click="$save = 1; $$post('{{ spark.url('some/template.twig') }}" would also work (although point 2 above is still an issue). I just don’t really like the idea of having multiple ways of doing things without a clear use-case for each (beyond just personal preference).

Namespacing the store

I really like the idea of injecting a store object into the template rather than injecting the variables directly.

Automatic updating of the store

If I inject a store object into the template then I can give it the method store.set(name, value) that will prevent users having to do |merge gymnastics.

The name

Not sure about this one. I’ll have to let it sit.

[khalwat]

1. nah, I was going to have you add the quotes in the method result to avoid that

2. People could simply add it themselves in the second parameter, e.g.: data-on-click="$$post('{{ spark.renderTemplate('...'), { csrfToken: whatever} }}')" no? Then they get to decide when/where they need it

3. Yeah don't add to the API, take away from it. I don't think the verbs are doing much other than obfuscating things

I really think that any quibbles with what's better/less verbose, etc. are far outweighed by a direct correlation with how Datastar works, and the fact that you leverage the Datastar documentation, example code, and knowledge.

And ya, meditate, release the attachment to the custom branding, and change the name. 🤣

Anyway my two cents. GL GL

[wbrowar]

I agree with @khalwat’s suggestion that instead of using spark.get() to use something like spark.renderTemplate(), then maybe pass in a method argument to specify GET, POST, etc... Sort of along the lines when we use fetch or axios in JavaScript.

From @khalwat’s comment:

People could simply add it themselves in the second parameter, e.g.: data-on-click="$$post('{{ spark.renderTemplate('...'), { csrfToken: whatever} }}')" no? Then they get to decide when/where they need it

I do like that Spark helps with the CSRF stuff, so having to get that yourself makes Spark less elegant, but maybe another Twig method might help. Something like, spark.renderOptions({ myOptions: goHere }).

Regarding the name and the API in general, I think it does make sense to either go all in on Datastar and make this plugin a way to do everything Datastar can do with some helpers to make the requests between the front end and back end easier. Another option might be to abstract all of this stuff and make an original reactive framework and just use Datastar under the hood to power it. I think the latter would be a ton of work and I'm not sure that was the original intent for Spark.

[bencroker]

Thanks for the input, Will! Please see the syntax changes in 0.0.5.

Another option might be to abstract all of this stuff and make an original reactive framework and just use Datastar under the hood to power it. I think the latter would be a ton of work and I'm not sure that was the original intent for Spark.

It is most definitely not.

[wbrowar]

Sweet I'll check it out!

[bencroker]

I just tagged 0.0.5 with some syntax changes.

Eliminating action verbs

I took your advice and it is now required to output $$get() yourself. The new way of outputting a Spark URL is using the sparkUrl() function:

<button data-on-click="$$get('{{ sparkUrl('_spark/main.twig') }}')">

I addressed the CSRF issue using a third argument.

<button data-on-click="$$post('{{ sparkUrl('_spark/main.twig', [], true) }}')">

Namespacing and updating the store

Datastar's store values can now be accessed and modified within templates rendered by Spark using the store variable. I know you’d prefer a fluid syntax for setting values, but I need to sit on it some more.

Username: {{ store.username }}

{# Modifies the value of `username` in the store. #}
{% do store.set('username', 'bobby') %}

{# Modifies multiple values in the store. #}
{% do store.setValues({ username: 'bobby', saved: true }) %}

The Dual APIs

I considered and even built out the tags {% spark fragment %}, {% spark redirect %}, etc. I thought it was a good idea too, but in building them I realised that tags are annoying as hell develop (Twig token parsing is 🤮), they are an unusual syntax that doesn’t feel natural, and (possibly most importantly for me) they don’t provide autocomplete.

I also want to abstract away Datastar’s event types and implementation. I want there to be no need for the user to understand what fragment or signal events are when working with Spark.

In the end, I opted to create separation of concerns by using sparkUrl() for your so-called “Frontend API” and spark.{function} for the “Backend API” (spark.remove(), spark.redirect() and spark.console()).

[khalwat]

I respectfully and wholeheartedly disagree.

That's fine, your project! Datastar CEO agrees tho :)

I’m still not convinced this is a good syntax. You cannot, for example, modify entry values using entry.fieldHandle('New title'). Instead, you have to use the more explicit entry.setFieldValue(fieldHandle, 'New title')

It's pretty standard fluent model syntax that's been around for ages: https://dev.to/mofiqul/fluent-interface-and-method-chaining-in-php-and-javascript-251c

You can change values with things like

        $query = (new Sql())
                ->select(['foo', 'bar'])
                ->from('foobar', 'f')
                ->where('f.bar = ?');

This is done all over in Craft and elsewhere... the only reason it doesn't work with custom fields for Entry objects (and I'd argue that it should work there too) is P&T didn't do it.

All of Craft's config settings are Fluent models now too: https://craftcms.com/docs/5.x/reference/config/general.html

Why force them to type {% send fragment %} (and learn what it means) when there’s no requirement to? And if one-to-one mapping to how Datastar works, users would need to use {% send signal %} to update store values. I see these as implementation details that can and should be abstracted away from the user.

That's fair, because a whole lot of value is added with the store magic implementation, and you could argue that for the standard case where most of the time you just want to send back a fragment, it's nice that they don't have to type anything.

I still think it makes the other events that you send up a little weird in syntax, but given that you won't be doing that most of the time, I'm coming around to the way you're doing it being a consistency compromise that's worth it.

I concede on this point, your way is better.

[bencroker]

That's fine, your project! Datastar CEO agrees tho :)

One of the goals of Spark is to remain as decoupled from the JavaScript library that provides the front-end functionality as possible (unlike with Sprig + htmx). With the current implementation of Spark, there is be nothing stopping me from swapping Datastar out for some new DataAsterisk JS library that comes out or, god forbid, forking Datastar and maintaining my own version.

Another reason is that I don’t want to stake a claim on Datastar. As unlikely as it sounds, what if someone else wanted to make their own Craft plugin that uses Datastar under the hood? Plus, I don’t want to even think about the confusion that would ensue when people visit the https://data-star.dev docs looking for how to do something in Craft Datastar.

So, hopefully, based on all of that, you can see why it makes zero sense to me to name the plugin “Craft Datastar”.

It's pretty standard fluent model syntax that's been around for ages

To be clear, the store variable already provides a fluent model syntax.

{% do set store.set('title', 'New title').set('count', 0) %}

Regarding the magic setter, it is important to distinguish between things like a Query class, in which select(), from() and where() are methods on the class, and the store’s values, which are dynamically set during runtime execution. Craft’s GeneralConfig has methods for setting each of its config settings. And even in your MetaGlobalVars class, only existing public properties can be set using the syntax seomatic.meta.seoTitle("Some Title") (and you could have also created methods for setting each property).

With the store, all values are dynamically assigned during runtime and stored in a values array. There are no class properties to assign values to. So while it is possible to make the syntax store.xyz(value) work, xyz is completely arbitrary and no static analyser can be aware of it, resulting in PhpStorm (understandably) complaining.

Whereas with store.set('xyz', value), PhpStorm provides autocompletion and displays the method arguments.

[khalwat]

You make no gains with the store.set() syntax, because you still get no autocomplete on what you're complaining about... 'xyz' does not autocomplete, and now you have more finicky syntax that's harder to type, and easier to make mistakes with.

{% do store.xyz(value) %}

is a lot easier to type, and less error-prone than:

{% do store.set('xyz', value) %}

...and your autocomplete argument is pointless, because the only thing it's autocompleting is a method you don't even have to type with the first example. You still get no autocomplete on the thing you claim to really want, 'xyz'.

What's the point in getting auto-complete on .set()? I'd rather just not have to type it at all.

Also at least with the store.xyz(value) syntax I can create PHP [#Attribute] classes to get autocomplete... whereas with store.set('xyz', value) I have zero options for ever getting autocomplete.

I'll probably just fork Spark and add the syntax in there, and use the fork. :)

[khalwat]

One of the goals of Spark is to remain as decoupled from the JavaScript library that provides the front-end functionality as possible (unlike with Sprig + htmx). With the current implementation of Spark, there is be nothing stopping me from swapping Datastar out for some new DataAsterisk JS library that comes out or, god forbid, forking Datastar and maintaining my own version.

Can we wager on whether either of these unlikely events ever happens?

I think you had the idea for an abstracted API, and gave it a name... and then realized that you really can't have an abstracted framework that layers on top of Datastar, because the layer you're providing is thin, and a whole lot of the code people will be writing using "Spark" will be super tightly coupled to Datastar.

So even if any of the unlikely events you discuss above happen, the layer you're providing isn't going to insulate the projects using it at all, unless you effectively write a Datastar clone.

It reminds me of the layer that P&T added on top of Yii for Craft 2 -- which they then abanded completely for Craft 3, realizing that they should embrace the framework instead.

Another reason is that I don’t want to stake a claim on Datastar. As unlikely as it sounds, what if someone else wanted to make their own Craft plugin that uses Datastar under the hood? Plus, I don’t want to even think about the confusion that would ensue when people visit the https://data-star.dev/ docs looking for how to do something in Craft Datastar.
So, hopefully, based on all of that, you can see why it makes zero sense to me to name the plugin “Craft Datastar”.

You realize you're talking to the author of Craft Vite, right? :)

So no, I'm not buying the arguments. But like I said, in the end, it's your project. You wanna call it Spark, Spark it is!

[bencroker]

You make no gains with the store.set() syntax, because you still get no autocomplete on what you're complaining about... 'xyz' does not autocomplete, and now you have more finicky syntax that's harder to type, and easier to make mistakes with.

I can see you’re not backing down on this one, so either you’re really trying to prove a point or you’re passionate about this! I’m going to assume it is the latter, and grant you your wish. Both syntaxes will be valid for the next release.

Can we wager on whether either of these unlikely events ever happens?

No. I’ve learned my lesson at least twice and will not wager with you again in future (that’s probably not true).

[bencroker]

I just tagged 0.0.6 with some syntax changes.

The third argument for sparkUrl() is now an optional method type (defaults to get if not provided).

<button data-on-click="$$post('{{ sparkUrl('_spark/main.twig', [], 'post') }}')">

This is because I’ve reintroduced a short-hand version spark() that makes typing $$get('') optional (I wince every time I have to), and under the hood, it calls sparkUrl().

<button data-on-click="{{ spark('_spark/main.twig', [], 'post') }}">

I’ve also reintroduced the {% fragment %} tag, but it is purely optional and should only be used if providing fragment options. I’ll document it once some changes to the Datastar API go through.

[khalwat]

If you really hate typing $$get() that much, I have bad news for you about using the rest of the Datastar API (which you're going to need to use anyway) 🤣

[bencroker]

I don’t see most people using any of the action plugins regularly (nor creating their own), so $$get() and $$post()are the only times the syntax $$fn() is likely to be used, and the spark() function just makes that optional.

[bencroker]

Tagged 0.0.7 with support for the store.xyz(value) syntax and renamed the plugin to... nah, just kidding, it’s staying Spark!

[khalwat]

❤️

I can live with Spark 🧨🤣

[khalwat]

Bless your heart @bencroker all is well with the world now:

Just need to add a little stub class:

<?php

namespace attributes\stores;

use Attribute;
use putyourlightson\spark\models\StoreModel;

#[Attribute]
class AutocompleteSearch extends StoreModel
{
    /** @var string $search The string to search for */
    public string $search;
}

...and then in your template:

{# @var store \attributes\stores\AutocompleteSearch #}

I have achieved inner peace 🧘