Process flow for HAL-FORM to support dynamic forms

[gillarramendi]

Basically we have tried to follow the proposed process flow approach in the spec and we didn’t found a solution to deal with, when the URL of the form definition is dynamic.
In other words, the current proposed approach uses the rel as a URL. It means that the clients must know the rel, (I guess that the rel must be static) and does not work properly when the form content is dynamic and changes constantly (this URLs can not be inside the client).

For instance in the following example we can see the URL to prepare the form for a specific order:

{
  "_links": {
    "self": {
      "href": "http://api.example.org/orders/1",
    },
    "http://api.example.org/orders/1/prepare-edit": {       ← url to obtain the hal-form
      "href": "http://api.example.org/orders/1/edit",
    },
  }
}

One possible solution for this case, could be to use another approach where we have independent rels, one for form preparation and another one for the submit, for instance:

"_links" : {
    ...
    "prepareForm" : {
      "href" : "http://api.example.org/orders/1/prepare-edit"    ← Url to create dynamic form
    },
    "submitForm" : {
      "href" : "http://api.example.org/orders/1/edit"
    }
  }

This solutions works well but the client don’t see the relation between both rels in the HAL response. In addition within our requirements (related to security), and once we don’t have the action URL within HAL-FORM, we need to infer this relation in some way to validate the incoming submit using the form definition.

This relation between both rels can be useful as well when we want to define the FORM in HTML format. I mean once we define action URL within HAL (the default behaviour) we don’t need to define again when we define the form. It’s important to note that HAL founder Mike Kelly likes HTML based form definitions.

So we have been thinking in a more suitable approach where the client can understand the relation between both rels and any tool or library can infer the relation between both rels as well. Basically we think that type property inside the link (from HAL spec) can be useful for that. For instance using this approach the HAL response:

"_links": {
      "edit": [{
          “type” : “application/prs.hal-forms+json”,
          "href" : "http://api.example.org/orders/1/prepare-edit"    ← Url to create dynamic form
      },{
          "href" : "http://api.example.org/orders/1/edit"
      }]
}

The resulting hal-forms response:

{
  "_links" : {
     "self" : {
      "href" : "http://api.example.org/orders/1/prepare-edit"
    }
 },
  "_templates" : {
    "default" : {
      "title" : "Edit",
      "method" : "put",
      "properties" : [
        ….
      ]
    }
  }
}

In that way we solve both requirements. We can deal with dynamic URLs for form definitions and we can express the relation between both rels improving the readability for humans and tools processing for client and server sides.

What do you think about this approach? Do you think it could be included in the HAL-FORM spec suggested process flow as the standard way to deal with this kind of dynamic forms?

[mamund]

@gillarramendi

first, i will tell you that this looks way too complicated to me. i suspect either a) you're over-engineering this or, b) i am just too thick to figure out your special case.

let's see ....

the current proposed approach uses the rel as a URL. It means that the clients must know the rel, (I guess that the rel must be static)

why would the rel from your services be unpredictable? in HAL, the rel is the key the client uses to get something done. are you planning to introduce previously unknown rel values? why do this?

rel, (I guess that the rel must be static) and does not work properly when the form content is dynamic

a repeated rel (that is an URL for a HAL-FORM) doesn't REQUIRE the HAL-FORM response be the same for all contexts. for example if i am logged in as an admin user, the HAL-FORM returned by a call to /api.example.org/rels/customer/123 might contain different template information than if i was logged in as a guest user.

(this URLs can not be inside the client).

right - the target URL for the action is always in the HAL document. it would be even if you were not using the HAL-FORMS extension. and there is nothing the the HAL documentation that REQUIRES you to make sure all target URLs for all users at all times must be IDENTICAL. again, if i am logged in as an admin user, the URL returned for a single order might be /orders/1/edit-admin and if logged in as a typical user it might be /orders/1/edit-user and, if i am logged in as a guest the LINK might not appear at all. all of this is possible w/o changes to HAL or to HAL-FORMS. what scenario is not covered here?

One possible solution for this case, could be to use another approach where we have independent rels, one for form preparation and another one for the submit

now it looks like you want to modify HAL responses, is that right? if so, you need to take this up w/ MikeKelly, not me.

This relation between both rels can be useful as well when we want to define the FORM in HTML format.

now you want HAL-FORMS to return an HTML document instead of a HAL document? if that's the case, you need to create your own extension (HTML-FORMS?).

Basically we think that type property inside the link (from HAL spec) can be useful for that.

again, this looks like making mods to the HAL spec and i'm not the person to be talking to about that.

Look, with HAL-FORMS you have the possibility to return any FORM instructions for any HAL link:href and there is absolutely no restrictions on what algorithms you use to do that. i've even agreed to the notion of including an unlimited number of templates in a single HAL-FORMS response.

i can't see what scenario you are unable to support with this kind of freedom.

help me out here. what's not possible with the HAL-FORMS spec that you MUST have?

[gillarramendi]

First of all, I would like to clarify that we are not requesting any change in the HAL-FORM response. We are just talking about the suggested process flow included within HAL-FORM spec, that actually is just a proposal and we know that is not mandatory.

why would the rel from your services be unpredictable? in HAL, the rel is the key the client uses to get something done. are you planning to introduce previously unknown rel values? why do this?

We don't want to do that, actually the opposite. We want predictable rel and unpredictable URLs, let me explain below.

Within the current process flow proposal included within the HAL-FORM spec, as far as we know the rel implements two functions:

• It is the rel: as you said is static and the clients will use it and they need to know the its value.
• It is a URL at the same time: that's the problem. It is a URL. but it can not change because is a rel at the same time.

We have problems with the second part, basically because it does not follow the hypermedia idea where the rel is static and the URL can be anything (dynamic) and the client should not know it.

Within our requirements we need a dynamic URL, why? Because the form content depends on 2 things:

• The URL: where ids (order id, item id, etc) are included, depending on which, the content of the form can be different. For instance, order 1, may not be editable depending on its state.
• The user context within the server (the roles, the session status, etc)

Let me show you a real use case ...

We have two Order resources:

Order 1:

{
  "_links": {
    "self": {
      "href": "http://api.example.org/orders/1",
    },
    "http://api.example.org/rels/orders/1/prepare-edit": {
      "href": "http://api.example.org/orders/1/edit",
    },
  }
}

Order 2:

{
  "_links": {
    "self": {
      "href": "http://api.example.org/orders/2",
    },
    "http://api.example.org/rels/orders/2/prepare-edit": {
      "href": "http://api.example.org/orders/2/edit",
    },
  }
}

As you can see, we have two different urls to obtain the HAL-FORM document:

• http://api.example.org/rels/orders/1/prepare-edit
• http://api.example.org/rels/orders/2/prepare-edit

These two urls can respond quite different HAL-FORM documents, depending the state of each Order:

Response of http://api.example.org/rels/orders/1/prepare-edit:

{
  "_links" : {
     "self" : {
      "href" : "http://api.example.org/rels/orders/1/prepare-edit"
    }
 },
  "_templates" : {
    "default" : {
      "title" : "Edit",
      "method" : "put",
      "properties" : [
        {"name" : "desc", "value" : "Order 1",  “readOnly”= “false”, "prompt" : "Description"},
        {"name" : "paid", "value" : "false", "prompt" : "Paid"}
        ...
      ]
    }
  }
}

Response of http://api.example.org/rels/orders/1/prepare-edit:

{
  "_links" : {
     "self" : {
      "href" : "http://api.example.org/rels/orders/2/prepare-edit"
    }
 },
  "_templates" : {
    "default" : {
      "title" : "Edit",
      "method" : "put",
      "properties" : [
        {"name" : "desc", "value" : "Order 2",  “readOnly”= “true”, "prompt" : "Description"},
        {"name" : "paid", "value" : "true", "prompt" : "Paid"}
        ...
      ]
    }
  }
}

The first Order has not been paid yet, so the desc attribute is modifiable, on the other hand, the second Order has been paid, so the desc is not modifiable.
Another option for the second case is not to include desc attribute in the form, but the problem is the same.

As you can see, the two hal-form documents are different, depending on the state of the order, in this case, if it has been paid or not.

Same thing happens with the suggested values of a field. They can change depending on the current state of the resource.

This approach works and we think it is aligned with hal-form specification requirements, but the rel links has become unpredictable, because we have to add the Order id into the url.

• http://api.example.org/rels/orders/1/prepare-edit
• http://api.example.org/rels/orders/2/prepare-edit
• One per order...

This is the main reason of our process flow proposal for dynamic forms. We prefer to have two urls, one to prepare the form and the other for the action.

a repeated rel (that is an URL for a HAL-FORM) doesn't REQUIRE the HAL-FORM response be the same for all contexts. for example if i am logged in as an admin user, the HAL-FORM returned by a call to/api.example.org/rels/customer/123 might contain different template information than if i was logged in as a guest user.

Yes sure I agree on that, but if the response depends on the URL content I don’t think so. For example a url like that: http:/api.example.org/customer/123/item/100

Right - the target URL for the action is always in the HAL document. it would be even if you were not using the HAL-FORMS extension. and there is nothing the the HAL documentation that REQUIRES you to make sure all target URLs for all users at all times must be IDENTICAL. again, if i am logged in as an adminuser, the URL returned for a single order might be /orders/1/edit-admin and if logged in as a typical user it might be /orders/1/edit-user and, if i am logged in as a guest the LINK might not appear at all. all of this is possible w/o changes to HAL or to HAL-FORMS. what scenario is not covered here?

Please see the examples presented previously

now it looks like you want to modify HAL responses, is that right? if so, you need to take this up w/ MikeKelly, not me.

No, we don’t want to modify the HAL response, we just want to document the process flow to deal with dynamic URL requirement.

now you want HAL-FORMS to return an HTML document instead of a HAL document? if that's the case, you need to create your own extension (HTML-FORMS?).

I’m sorry I didn’t explain it properly. We were saying that this idea could be useful for other form support formats. Within the Spring-HATEOAS extension we are working on with @dschulten we want to support more than one form format, but maintaining the part based on HAL. In my case I was saying just HTML, an HTML form.

again, this looks like making mods to the HAL spec and i'm not the person to be talking to about that.
Look, with HAL-FORMS you have the possibility to return any FORM instructions for any HAL link:hrefand there is absolutely no restrictions on what algorithms you use to do that. i've even agreed to the notion of including an unlimited number of templates in a single HAL-FORMS response.
i can't see what scenario you are unable to support with this kind of freedom.
help me out here. what's not possible with the HAL-FORMS spec that you MUST have?

I hope that the new explanation can communicate better our thoughts.

Thanks!

[mamund]

@gillarramendi

ok, let's take this one step at a time:

We want predictable rel

then we start there -- no more examples of rels that are unpredictable.

now, if you have various "states" to support for an order, why not simply emit rel values that reflect this information directly?

{
  "_links": {
    "self": {
      "href": "http://api.example.org/orders/1",
    },
    "http://api.example.org/rels/paid-order": {
      "href": "http://api.example.org/orders/1/edit",
    },
    "http://api.example.org/rels/unpaid-order": {
      "href": "http://api.example.org/orders/2/edit",
    },
  }
} 

does this solve your problem? if not, what is still unsupported?

let's stop right there before continuing.

[gillarramendi]

Ok, let's take this example as a base.

Would you mind clarifying your example?
On one hand the self suggests that we are seeing the link for order/1, but then we see a unpaid-order link to order/2 which makes us thought that it is a list and selfrel should be /orders/ instead.

[mamund]

{
  "_links": {
    "self": {
      "href": "http://api.example.org/orders/",
    },
    "http://api.example.org/rels/paid-order": {
      "href": "http://api.example.org/orders/1/edit",
    },
    "http://api.example.org/rels/unpaid-order": {
      "href": "http://api.example.org/orders/2/edit",
    },
  }
} 

OR

{
  "_links": {
    "self": {
      "href": "http://api.example.org/orders/1",
    },
    "http://api.example.org/rels/paid-order": {
      "href": "http://api.example.org/orders/1/edit",
    }
  }
} 

OR

{
  "_links": {
    "self": {
      "href": "http://api.example.org/orders/2",
    },
    "http://api.example.org/rels/unpaid-order": {
      "href": "http://api.example.org/orders/2/edit",
    }
  }
} 

[gillarramendi]

Sounds good, so to be sure that we are all on the same page which would be your suggested approach for N possible different states? (for example SENT/UNSENT & ABROAD/INLAND, we could potentially have a lot of variants)

[mamund]

@gillarramendi

sorry - was away from the machine for a bit.

first, N possible sounds like a trap!

i think the Q i have for you now is what is your anticipated set of transitions? when i build Web APIs the first thing I do is map out the list of possible transitions (using ALPS, actually). have you done this for some sample apps? what is your count?

second, let's drop HAL-FORMS for a bit and focus on the challenge (not the solution). what is it you really want to be doing here? what is your problem space?

i am very interested in this thread -- you are working on an important implementation set and are apparently running into serious problems. this convo has the potential to make an important impact on the design of HAL-FORMS and i love that.

the challenge for me is that i'm not in the room with you. i don't know your problem space and am having a hard time getting that from your responses so far. please stick w/ me on this and help me understand your use cases. I think this can help quite a large number of ppl once we sort it out.

to that end i'd be happy to do a google chat or even video hangout if you think that will make it easier for you to bring me up to speed on this and help you get to a good conclusion.

cheers.

[mamund]

@gillarramendi

added a gitter room (slack-ish thing) if you want to chat live there.

[anderruiz]

Thanks for the response I believe we are moving forward with this issue!! :). I'm Ander, @gillarramendi colleague at HDIV and I'm also starting to work on our HAL-FORMS implementation

Let’s try to explain what we are trying to achieve, we are a company that provides solutions to fight web applications security risks. We are now changing our focus from traditional web applications to services, and we chose HAL-FORMS to be our reference implementation for REST services.

Our goal is to present this architecture as the way our clients should develop their REST services. Our clients are heterogenous and some of them have difficult process workflows including entities with several “editing” states. So whichever solution we choose, it needs to provide a way to handle them. So at this point we don’t have a problem but we still need a solution :)

As you said the “N situation” is a trap because we will require N! pseudo-states that the client will need to know and therefore, it will imply coupling the editing rules in some way in the client because it needs to be aware of that and in our vision that is not the way to go.

So, from a high level point of view what is what the client should know?, In our opinion the REL should clearly define the operation, in this case EDIT, and the client should not be aware of the state, or the options/fields that should be enabled. With that REL that clearly define the operation the client should go to a dynamic link in which the appropriate HAL-FORMS document for one particular order could be retrieved, with that definition and the SUBMIT URL the client could post the form back to the server.

So ideally would something like

REL -> "http://api.example.org/rels/orders/edit"
Real URL (with templates) for the HAL-FORM document ->"http://api.example.org/rels/orders/{id}/form"
Real URL (with templates) submitting the form -> "http://api.example.org/rels/orders/{id}"

That would be the more flexible situation that could solve any problem as the form document would be customizable because in the server the ID could be known. But now, we need to do this in the best way that is possible, we have some ideas on how to do this, but perhaps the best approach if you have something in mind that could give us a flexible framework for our client to work with

Feel free to contact us by mail, hangouts or by skype if you prefer, I believe we can keep things moving faster in that way

[mamund]

so, what i keep seeing you and others repeat here is a desire to use two URLs for each REL. and i just haven't gotten to understanding why this keeps coming up.

now the "server knowing the ID" is interesting. i assume you mean the server returning the HAL-FORMS response knows the ID of the resource for which the HAL-FORMS response is being created. is that right? i'd like to know why this is important for you. what is this server that returns HAL-FORMS doing with the resource ID? why is this helpful to you?

i;ve added you to the gitter channel for this repo so we can do a live chat but you'll need to respond to the invite. or just go here to get started: https://gitter.im/mamund/hal-forms

[dschulten]

I can't speak for the HDIV colleagues. But given #13 I can say that I wish I could emit a diifferent form for a different item, e.g. in a list of items contained in my resource.

An example from one of my projects:

A user can get a list of products he bought from a company. Products can be sold from different subisidiaries. The customer wants to request a callback via phone from the place where he bought his product. The subsidiaries have different opening hours.

So the form for a product should let the user choose a time today when they want a callback on the phone from the guys who sold them this thing. After 4 pm, timeslots for tomorrow should be offered, too.

Obviously in the afternoon the morning timeslots should not be available. If an office is closed from 12 am to 4 pm, those timeslots should be filtered out, too.

It seems this is not a big issue if you have no forms with suggested values (I am not aware of a hypermedia format other than html which officially has suggested values). With suggested values, the itch starts for me.

In the example above, the challenge is to populate a hal form with a list of eligilble timeslots, equivalent to this form:

Current time: 11:15 am

Call me back, please
Name: ___
Phone No: ___

Product: Watchamacallit 2000
Seller: Wise Guys, Big City
[ ] 11 am - 12 am
[ ] 4 pm - 5 pm < -- after siesta
[ ] 5 pm - 6 pm

Product: Wombat XL
Seller: Pet Shop, Small Town
[ ] 11 am - 12 am
[ ] 12 am - 1 pm <-- shop closes early today

[ Submit ]

My idea is not that the hal-form has to contain all this. The question is, can we design a flow which allows to contextualize the suggested items.

In part I can contextualize:

// from api entry page
get /products (rel ex:products)
get /sellers/wiseguys (rel ex:seller on product)
get /rels/callback-request (rel ex:callback-requests) <--  describes hal-form POST request
post /sellers/wiseguys/callback-requests (rel ex:callback-requests)


name="Dietrich"&phone="+49 1727665554",slot="11am"&slot="5pm"

-> challenge for the hal-form: how to find out the callback opportunities for wiseguys?

Observation: when the client requests the hal-form, it knows for which target url it requests the hal-form. If the client could tell the server the href for which it requires the hal-form, that could prove helpful.

HAL response

_links: { 
  "ex:callback-requests": [{
    "href" : "http://example.com/sellers/wiseguys/callback-requests" <-- client knows it wants this
   }, {
     "href" : ...
   }]
}

Best,
Dietrich

[mamund]

interesting. what you want is to be able to pass some context information from the HAL document when you make the request for the HAL-FORMS document. more than just the REL, right?

so, you're making a request for a HAL-FORM document that will be associated with a HAL URL, right?

how about this:

HAL DOC RESPONSE
  "curies": [
    {
      "name": "ex",
      "href": "http://api.example.org/rels/{rel}",
      "templated": true
    }
  ],
  "_links": { 
    "ex:callback-requests": [{
     "href" : "http://example.com/sellers/wiseguys/callback-requests"
    }
  }]
}

HAL-FORMS REQUEST
GET http://api.example.org/rels/callback-requests?href=http://example.com/sellers/qiseguys/callback-requests

is that what you need? or do you need something else?

keep in mind HAL-FORMS already supports templated as a feature. that allows clients to fill in the returned HAL-FORM with state information known to the client.

the question here is whether you want HAL-FORMS to be telling the client some state data or if you want HAL-FORMS to allow clients to supply their own state data in selected locations.

does this make sense?

[mamund]

@anderruiz @gillarramendi

would love to get your feedback on my last post to this thread.

is this what you're looking to do?

[dschulten]

At least for spring-hateoas and its future hal-forms support that might be just what we need, at least for the situations I have in mind. As I said, I don't know if that covers it for HDIV.

We need to come up with a way how we can support the suggested client workflow, and there were some ideas already. I'll look into it, based on your suggestion.

[mamund]

if this seems valuable, i can suggest a slight mod to the Suggested Client Processing section of the spec.

I'd include a MAY for passing the associated URL of the REL in the HAL-FORMS request using {?href}

[robertovelasco]

Thank you very much for your example @dschulten, that’s exactly the requirement, well done!
Thank you Mike as well for your support, we really appreciate!

First of all we would like to answer Mike question.

When you say

assume you mean the server returning the HAL-FORMS response knows the ID of the resource for which the HAL-FORMS response is being created. is that right? i'd like to know why this is important for you. what is this server that returns HAL-FORMS doing with the resource ID? why is this helpful to you?”

That’s right and that’s exactly the example explained by @dschulten. I mean, related to the example, when we have this specific seller, /sellers/wiseguys/, the form content depend on the specific seller “wiseguys” in that case, so the URL to obtain the form should contain the id and must be different for each specific seller. That’s why we need specific URL for the preparation of each form. So basically we need 2 URL per each action:

• Form Preparation URL
• Submit URL

I think at this point we know what we need and why we need, the question now is how to implement those requirements.

As far as we understand, @dschulten has proposed that the clients adds an additional parameter to the form preparation url, to give the contextual information to the server (wiseguys) in the example. Honestly we don’t like this implementation strategy. First we consider that if we send the contextual information within an additional parameter may complicate the server side code.

Secondly and even more important, we consider that it is much more cleaner if the server sends 2 URLs (form preparation & submit) per action to the client directly. We consider it follows the hypermedia principle where the server defines the URLs all the time. Basically that’s what HTML does.
Don’t you think that it would be easier to understand when you read the HAL response from the server?

So if we use the approach based on 2 URL that come from the server, how can we implement that? How can we send 2 URLs to the client? We have already proposed some ideas in previous emails but we are open for other ideas:

"_links": {
      "edit": [{
          “type” : “application/prs.hal-forms+json”,
          "href" : "http://api.example.org/orders/1/prepare-edit"    ← Url to create dynamic form
      },{
          "href" : "http://api.example.org/orders/1/edit"
      }]
}

Additionally a more formal approach as proposed, has another indirect advantage from our perspective: tooling. Let me explain why.

Imagine that we want to develop a tool that browses a HAL & HAL-FORM API (I think we should do that), as we do with HTML hypermedia format. So we want to render the form (graphically or using a console) and let the user to fill the different fields. How can we know which is the submit URL? We can not.

So we think would be advisable to have a more formal approach to define the relation between the URL to prepare the form and the URL to submit form and of course we have 2 URLs per action to cover all possible cases, including the cases where the values or fields of the form depend on the URL content.

[dschulten]

@anderruiz @robertovelasco @gillarramendi my idea goes along these lines:

@RequestMapping(value={"/sellers/{seller}", "/rels/callback-requests"}
public Resource<Seller> getSeller(@PathVariable String seller) {
  sellerResource.add(linkTo(methodOn(this.getClass())
    .addCallbackRequest(new CallbackRequest(customerName, customerPhoneNo, 
        seller, Collections.emptyList()))
    .withRel("callback-requests");
  return sellerResource;
}

@RequestMapping(value={"/sellers/wiseguys/callback-requests"}
public void addCallbackRequest(@RequestBody CallbackRequest request) { ... }

public class CallbackRequest {
  @JsonCreator
  public CallbackRequest(
    @JsonProperty String name, 
    @JsonProperty String phoneNo, 
    @JsonProperty String seller,
    @JsonProperty @Select(options = TimeslotOptions.class, args = "seller")  
        List<String> timeslots) {}
}

Note that getSeller both serves the request for seller and the rel request. What I hope now is that in HalFormMessageConverter for requests to prs.hal-forms+json, we can do the following (pseudo code):

protected void writeInternal(Resource resource, HttpOutputMessage outputMessage) {
  // use RequestContextHolder.getRequestAttributes() to get 
  // "/rels/callback-requests?href=" from request
  String rel = getRelFromRequest(); 
  // now rel="callback-requests", using default curie provider to derive the rel
  String href = getHrefFromRelRequest();
  List<Affordance> affordances = (Affordance) resource.getLinks();
  Affordance affordance = findAffordanceWithRelAndHref(rel, href);
  // render affordance as hal-form  
}

Not quite sure if we get the request url from RequestContextHolder, I'll have to check tomorrow.
But what do you think of the general approach:

• allows two-step hal-form flow
• still we code the Affordance once, but dynamic.
• the need to be dynamic justifies that we get the seller data twice, use memcached or redis to reduce load on backends

PS. maybe we can avoid the need to add the "/rels/..." urls everywhere by automatically making the seller method responsible for the rels it emits when it is called for the first time. Need to ask Oliver.

[dschulten]

As far as we understand, @dschulten has proposed that the clients adds an additional parameter to the form preparation url

I think Mike came up with that. I only noted that the client knows what the server would like to know at that point.

First we consider that if we send the contextual information within an additional parameter may complicate the server side code.

Please explain that. The incoming href would be used for plain string comparison. What do you see as potentially complicated?

Secondly and even more important, we consider that it is much more cleaner if the server sends 2 URLs (form preparation & submit) per action to the client directly.We consider it follows the hypermedia principle where the server defines the URLs all the time. Basically that’s what HTML does.

Hm, but in HTML the server does not send two URLs.

If the client tells the server for which resource it needs that hal form, the server still controls its uri space. The client does not know any uri in advance, and it won't break if uris change. The extension rel uri must be kept stable, of course.

On the other hand, if hal-forms only works if I rework my original hal-response by adding links to hal-forms, then hal-forms are not a "transparent overlay" over hal. That was part of why hal-forms seemed elegant to me, they add metadata without changing the resource. Having form links adds links to the resource itself, not metadata. To illustrate this: the links would also be present in a Siren or Xhtml presentation of the resource.

I am not saying that everything is solved already. Sometimes rel uris cannot be served by the API provider, e.g. IANA or possible 3rd-party extension rel uris. But that is a separate problem.

BTW: links can also come as metadata, rfc5988.

Best,
Dietrich

[dschulten]

Not quite sure if we get the request url from RequestContextHolder, I'll have to check tomorrow.

It turns out that a http message converter can access the request URL in its write/writeInternal method:

RequestAttributes requestAttributes = RequestContextHolder.getRequestAttributes();
if(requestAttributes instanceof ServletRequestAttributes) {
  HttpServletRequest request = ((ServletRequestAttributes) requestAttributes).
  String pathInfo = request.getPathInfo();
  String queryString = request.getQueryString();
}

Using the CurieProvider to reconstruct the rel works, too.

Adding rel URIs to the methods which serve the resource having those rels would not be specific to hal-form. The same rel URI could be used to serve a JSON-Schema or a generated html description for the actions that are possible with the href marked by the rel.

This does not decide anything, but I wanted to note that my idea is feasible.

[dschulten]

Discussion with @anderruiz has shown that my idea has a flaw: it requires to replay the request for the initial hal response, and that won't work in the hal-forms request phase, armed only with the rel url and the rel target href.

Furthermore, the rel target href does not give me the necessary context to build the form metadata at all. What we need to build the form is the context of the hal request phase.

The client knows that context, too. It is the URL of the hal-response. I just want to mention it. I need a few experiments to see where this leads.

[dschulten]

It is possible to solve this if we get both the src href of the current hal response and the destination href of the rel. The src href is necessary to establish the necessary context, The dest href is necessary to distinguish rels on multiple items.

@mamund: would you be willing to add a MAY for passing the associated URL of the REL in the HAL-FORMS request using {?dest} and the URL of the current HAL response using {?src} ?

@anderruiz: We can use a filter to forward to src and then use a message converter to look up the link with matching rel and destination href. The spike is already working,

I'll put my spike into https://github.com/dschulten/hydra-java.

Again, I do not think all problems are solved. E.g. I have found an issue with rels that use fragment identifiers, as in http://api.example.com/rels#orderedItem. When the hal-forms client requests such a URI, the fragment identifier won't be sent to the server. That has nothing to do with #14, I just found it while working on the spike.

Another issue might be services which do not follow the PRG pattern for POSTs, but immediately return a POST response. I can imagine that this is an issue not only with my forwarding implementation, but also with manual approaches.

[mamund]

interesting proposal.

tell me more about this one:
"The dest href is necessary to distinguish rels on multiple items."

what does "distinguish rels on multiple items" mean/refer-to?

​

[dschulten]

Given the hal response below with an embedded event collection where each event has a POST form to add review comments, marked by rel ex:review:

{
    "_links": {
        "self": {
            "href": "http://localhost:8080/webapp/hypermedia-api/events"
        },
        "ex:search": {
            "href": "http://localhost:8080/webapp/hypermedia-api/events{?name}",
            "templated": true
        },
        "curies": [
            {
                "href": "http://localhost:8080/webapp/hypermedia-api/rels/{rels}",
                "name": "ex",
                "templated": true
            }
        ]        
    },
    "_embedded": {
        "eventList": [
            {
                "performer": "Walk off the Earth",
                "workPerformed": {
                    "name": "Gang of Rhythm Tour",
                    "_links": {
                        "ex:review": {
                            "href": "http://localhost:8080/webapp/hypermedia-api/reviews/events/1"
                        }
                    }
                },
                "location": "Wiesbaden",
                "eventStatus": "EVENT_SCHEDULED",
                "_links": {
                    "self": {
                        "href": "http://localhost:8080/webapp/hypermedia-api/events/1"
                    }
                }
            },
            {
                "performer": "Cornelia Bielefeldt",
                "workPerformed": {
                    "name": "Mein letzter Film",
                    "_links": {
                        "ex:review": {
                            "href": "http://localhost:8080/webapp/hypermedia-api/reviews/events/2"
                        }
                    }
                },
                "location": "Heilbronn",
                "eventStatus": "EVENT_SCHEDULED",
                "_links": {
                    "self": {
                        "href": "http://localhost:8080/webapp/hypermedia-api/events/2"
                    }
                }
            }
        ]
    }
}

As you can see, both ex:review links have a different href

• http://localhost:8080/webapp/hypermedia-api/reviews/events/1
• http://localhost:8080/webapp/hypermedia-api/reviews/events/2

For the sake of argument, let us assume each event has a different rating system, depending on the event organizer, and we want to render review forms with ratings for each event. The organizer of event 1 wants one to five stars. The organizer of event 2 would like to have its event rated with A to F. The server can find this out if has access to the event in question, hence the hal-form could spell out the list of possible values as suggest items.

In the case of spring-hateoas, the method that produces the response above returns a structure containing data with links (ResourceSupport.java). So it is not complicated to get the data structure again by forwarding the hal-forms request to src and look up the link with matching rel/dest combination. That is the link we must render as hal form. For the user, there is no special coding required to get hal-forms.

It can also be implemented manually by applying a UriTemplate to the incoming dest URL. Given a method which is responsible to render hal-forms, it could decide based on the rel which dest UriTemplates apply for this rel, find the matching UriTemplate, extract the variables and go find the event.

[mamund]

so, including the target/dest URLs in the HAL-FORMS request can help you?

i think you also mentioned including the source URL (the HAL response's self URL in the HAL-FORMS request, is that right?

what i see developing here is a need for some additional data-passing in order to support the implementation details of your framework. IOW, in order for the server-side to accomplish some goals, you want the client to send additional data/context.

i'm fine w/ this idea and what i am looking to do here is find a general algorithm to support this. IOW, something that is not implementation- or domain-specific.

it looks to me like you're asking for a way to describe (template) the HAL-FORMS URL, right? that feature already exists for HAL, right? (e.g. templated URIs for rel). sop you can do this w/o any changes to HAL-FORMS or HAL, i think.

[dschulten]

what i see developing here is a need for some additional data-passing in order to support the implementation details of your framework.

If it were framework-specific, it would not belong into the hal-forms spec. But admittedly, I'd like the solution to work with Spring Hateoas, too :-)

it looks to me like you're asking for a way to describe (template) the HAL-FORMS URL, right? that feature already exists for HAL, right? (e.g. templated URIs for rel). so you can do this w/o any changes to HAL-FORMS or HAL, i think.

Enticing :-)

The hal spec doesn't outright forbid it, it only says the token 'rel' is present.

CURIEs are established within a HAL document via a set of Link Objects with the relation type "curies" on the root Resource Object. These links contain a URI Template with the token 'rel', and are named via the "name" property.

So we would have something like:

"curies": [{
"name": "acme",
"href":
"http://docs.acme.com/relations/{rel}{?self,href}",
"templated": true
}],

I chose 'self' and 'href' as tokens here instead of 'src' and 'dest' because those names at least are mentioned in a hal response somewhere.

What worries me a bit is the fact that the tokens 'self' and 'href' used alongside 'rel' are not well-known to a client, so they have to be documented for the API.

Therefore, having them mentioned in hal-forms as a MAY would be helpful. But I can see that it is hard to prove that other frameworks might not have other requirements.

We'll have to evaluate what that means for our hal-forms impl, but I think we can go forward with this.

[anderruiz]

Thanks for your suggestions and help defining this matter. Sorry for not having answered earlier but we were analyzing one possible solution but at the same time we did not want to pollute the issue thread with something that may (again) not be a complete solution.

After thinking about it over the weekend we believe that Mike’s suggestion to use the {href} parameter as part of the HAL-FORM request is a possible solution and could solve our current issues.

Mike’s suggestion was to use this format

HAL-FORMS REQUEST

GET http://api.example.org/rels/callback-requests?href=http://example.com/sellers/qiseguys/submit-callback

** The qiseguys submit url was changed to clarify the explanation bellow

This could be achieved if the curie has this definition

"curies":

[
{
"name": "ex",
"href": "http://api.example.org/rels/{rel}/{?href}",
"templated": true
}
]

This provides enough information for the server to know how to create the appropriate form for that request, it is however strange (and cumbersome) to get that information from a parsed URL parameter. That’s why, while trying to find a simpler approach we came up with an idea that it could work. What about using this CURIE instead of the previous one?

"curies": [
{
"name": "ex",
"href": "{href}/{rel}", // Other options are possible like {href}{?rel}, {href}{&rel} an so on >//depending on what is best for the client needs
"templated": true
}
]

With this CURIE the HAL-FORMS request would be:

GET http://example.com/sellers/qiseguys/submit-callback/callback-requests

In our opinion this is a more natural way to process the information in the server, while keeping the original simplicity of Mike’s idea. What do you think about it?

Additionally we would like to have a explicit way of marking that a particular REL is a HAL-FORM so that we can provide tooling for it, for doing that we suggest taking advantage of the already defined CURIE and addition “type” parameter to it:

"curies": [
{
"name": "ex",
"href": "{href}/{rel}", // Other options are possible like {href}{?rel}, {href}{&rel} an so on
"templated": true
“type”: “application/prs.hal-forms+json”
}
]

We also appreciate your thoughts on this. As we did want to test this approach in a real situation we already modified HAL-BROWSER client to support this and a demo application to support the server side. Both changes were straightforward and validate the solution.

Anyway both solutions fit with the current spec and we consider just an additional comment in the documentation would be enough to cover those use cases. We would like to show this work in Spring I/O (19-20 May), with that in mind @mamund we would need your support at issue #9.

Of course, we will include the references to the issues created for this support and the people that has been involved in this work such as @dschulten and @mamund

I hope we can reach an agreement on this matter as soon as possible

[dschulten]

@anderruiz

it is however strange (and cumbersome) to get that information from a parsed URL parameter

Many servers parse incoming url requests to build their responses, there is ample tooling for this, and urls as query params are not uncommon either.

But I have some questions:

• what should hal-form aware clients fill in for href?
• If a client is not hal-forms aware, and expands the uritemplate {href}/{rel} with href=null, then the resulting extension relation type for a compact uri ex:callback-requests will be the context-relative URI /callback-requests, or, after making it absolute, http://example.com/callback-requests. Is that intended?
• a hal-form client dereferences the rel URIs on demand or prefetching and actively negotiates for a hal-form response. Why can your tooling not do the same?

[mamund]

@anderruiz @dschulten

as far as i can tell now, the suggested solution is centering around two things:

• using HAL CURIES to ensure the URL in the rel value used to request a HAL-FORMS document has all the state pointers needed to instruct the server on how to emit the appropriate HAL-FORMS response.
  and
• using the existing HAL type property of a link object to aid some tooling

on the face of it, i don't have any problems with these suggestions since they do not make any demands on the HAL-FORMS specification (correct me if i am wrong on this).

i do, however, have similar Qs as @dschulten on the implementation details. these are, of course, none of my affair and i leave it to you to work this out. i'd be happy to continue to assist in the discussion as needed, of course.

[anderruiz]

Many servers parse incoming url requests to build their responses, there is ample tooling for this, and urls as query params are not uncommon either.

It is way more strange to work with a parameter URL, as an example I've never worked in any project having URL parameters, it would be much more complicated to convince a client to work with that approach in my opinion, but as @mamund said we are only using HAL CURIES here so I suggest document this {href} approach as part of the suggested flow, we can even include both CURIEs as examples if you prefer,

"href": "http://api.example.org/rels/{rel}/{?href}",
"href": "{href}/{rel}", // Other options are possible like {href}{?rel}, {href}{&rel} an so on

what should hal-form aware clients fill in for href?

The href of the submit link, that is what @mamund suggested right?

If a client is not hal-forms aware, and expands the uritemplate {href}/{rel} with href=null, then the resulting extension relation type for a compact uri ex:callback-requests will be the context-relative URI /callback-requests, or, after making it absolute, http://example.com/callback-requests. Is that intended?

It is not intended but it could be a fallback URL. In this particular case if the client does not support HAL-FORMS it would not be able to navigate to the submit URL also, so, we could consider that situation out of the scope of the analysis.

a hal-form client dereferences the rel URIs on demand or prefetching and actively negotiates for a hal-form response. Why can your tooling not do the same?

If we do not (at least recommend) some sort of implicit definition of what is a HAL-FORM and what is not, we could end up having a mess. Imagine that a client support several standards similar to this, what would be the approach to start trying with different MIME types for one URL. We believe that even though we can use implicit RELs it would be better and easier to automate if we have some sort of annotation about that.

So in SUMMARY if we are to present this in the Spring I/O we should move this issue forward, in our view we should:

• Document the usage of CURIEs and {href} parameter (submit URL) as part of the suggested process flow in the spec (for those forms that need to be dynamic). We could even suggest both CURIE types, one that is more HAL compliant and the other that could make HAL-FORMS easier to implement for the clients
• It is not mandatory but we should mention the possibility of including "type" inside of the CURIEs, we could consider it OPTIONAL,

Thoughts?

[mamund]

@anderruiz @dschulten

from the HAL-FORMS point of view, the spec will not change I'll not be naming URITemplate variables in the spec. I will not include proposed templates to use for rel URIs, etc.).

servers are free to emit any rel, CURIE, and template they wish. HAL-FORMS will work just fine no matter the value supplied.

it will, however, be up to servers to emit HAL documents that contain the proper templates, CURIEs, and other state data in a manner that client apps can understand, resolve, and use at runtime.

i will add some text that instructs client apps who are unable to construct a valid URL from the rel value (including any template information) SHOULD simple ignore that rel with regards to HAL-FORMS. if course, if the URL constructed is valid, but points to a non-existent place, this results in a 404 and the client can resume as stated in the spec.

[dschulten]

@anderruiz the primary function of the rel is to guide the client in its decision what transition it wants to choose next. The rel says why the client would want to follow a link. The client looks out for IANA defined rels, or for extension rels defined by an API provider, which are URIs to make them unique on the internet. A curie is only unique after resolving the curie prefix to a full URI. At least in theory, a client does not follow a rel "ex:tasks", it first resolves the curie and then decides if the resolved URL is what it is looking for.

In order to fulfill this role, the extension rel URL cannot be dynamic. Making an individual href like http://api.example.org/tasks/1?rel=tasks an extension rel URL defeats the purpose of an extension rel. A client cannot be programmed to follow that rel, because there would be unlimited numbers of rel urls.

That is different for a curie template http://api.example.org/rels/{rel}{?href} which expands to http://api.example.org/rels/task for a client which is looking to resolve a curie to its unique form and expands rel as described by the HAL spec, but provides no value for href. The template can be written in such a way that the {?href} part goes away after expansion if the href variable is not specified.

Only a client prepared to expand other variables than the rel variable would be able to construct an individual URL for the hal-form. Still. the whole mechanism seems brittle. E.g. we must hope that existing clients use a proper UriTemplate expansion engine and do not just replace {rel}.

I do not think we have something solid enough to build dynamic hal-forms just yet. If the URI template variables and its handling is just a private extension, an API provider does not gain much by including it - the majority of hal-forms clients would not understand what is going on. I think we should stick to static forms for the moment, or maybe forms that behave differently for different users (that should already be possible based on the Authorization header).

[mamund]

@anderruiz @dschulten

one possible way to provide both dynamic choice and stable URL is to make the dynamic part a fragment-id (http://api.example.org/rels/update#this-is-the-fragment-id).

it is "playing" a bit w/ the content of the URL (it is dynamic content, but it is not part of the base URL).

this has some possible advantages:

• clients who understand the pattern but can't properly resolve the template will still have a valid URL to call for a HAL-FORMS
  ​
• clients who do not understand that pattern can ignore the fragment and still construct a valid URL for a HAL-FORMS call
• client who do not support HAL-FORMS can still use the URL as a guide for selecting the proper code to run (based on human documentation)
• services receiving client calls can be arranged to arrange in "default" ways when the HAL-FORMS request is made w/o the fragment-id present

BTW - the exact nature of the fragment-id needs to be reviewed (what is valid, what is not, can you smuggle the state info you need in this way, etc.)

[anderruiz]

We agree, I believe that for those situations in which you cannot control the type of clients that are going to access @mamund suggestion is perfect and would meet our needs. On the other hand when the clients that are going to target the API are known, there is more room for deciding the type of CURIE that is best suited for the project.

@mamund Would you include your suggestion in the spec?, it could be interesting, and it is 100% compatible

[mamund]

i would include some of the processing information but not anything suggesting how URLs are constructed. that's implementation detail that you are welcome to document and share yourselves, of course.

i'm adding the suggest matrerial today and, after that will fashion a PR for updating the processing language you can review and comment upon.

[dschulten]

But rfc 3986 says:

the fragment identifier is separated from the rest of the URI prior to a dereference, and thus the identifying information within the fragment itself is dereferenced solely by the user agent, regardless of the URI scheme

This means, the fragment identifier is not sent to the server, so the server cannot use it for anything. If the media type has a concept of fragment identifiers, the client can narrow down the fragment of the received resource @mamund: was that your idea? If not, I am afraid this won't work.

@anderruiz we must assume that a compliant hal-forms client tries to expand the curie template with a rel variable, and then dereferences the resulting url. To me it seems we should get that to work first, by all means.

[mamund]

@dschulten :

yeah - i got that backwards. sorry.

in the end, the URL constructed for the HAL-FORMS request will be the value of the rel property. how this value is derived (static or templated) is an implementation detail. it is up to the services to provide a template that can be successfully constructed by clients when relying on the algorithms in RFC6570.

i'll review the text for processing regarding constructing URLs from templates, but I suspect that text won't change much.

i think the challenge faced here is how to use the spec to create the implementation you want. the discussion here continues to revolve around passing state information to services when fashioning HAL-FORMS responses. AFAICT, there is more than enough computing power already in the spec to do that (e.g. through templated URLs).

it looks like the solution runs counter to some other design elements you want (e.g. static rel values) and that is a trade off you'll need to resolve in your implementation. there is nothing that i know of that can be written into the extension spec to make that trade-off for you.

[mamund]

@anderruiz @dschulten et al...

In shoring up some example for my upcoming book, i took another run at support for dynamic forms. What I decided to define was a set of OPTIONAL query parameter that client apps can use to pass information to the server when asking for a HAL-FORMS document. Here are the ones I defined so far...
http://rwcbook.github.io/hal-forms/#_the_hal_forms_query_string_registry

Note: I've forked the spec for the book (to make sure I don't break anything here) and that lives over here: https://github.com/rwcbook/hal-forms

Check these out and let me know what you think.

[anderruiz]

@mamund is there any chance to include the modifications I did in the Pull Request, for your book proposal?. On the other side I like your modifications for making HAL-FORMS specification more dynamic.

[mamund]

all: this has been up for quite a while w/o any additional feedback.

the changes for the query string: http://rwcbook.github.io/hal-forms/#_the_hal_forms_query_string_registry come close to handling most of the challenges brought into this thread although, IIRC, my solution is not the same as suggested and may not cover all the cases you pointed out.

i'm going to close this issue and if you like, we can start a new issue based on the query string registry support added to the spec.

thanks for all the work to make HAL-FORMS a better spec.