IIAs in index but deliberately not returned with get-response (to avoid und as language)

[demilatof]

We've just entered into production and we are doing some explorations.
As first try, we called IIA Index from uw.edu.pl (I mention them because they should be almost the reference implementations).
We received 4 IIAs, but only one can be read.
For the others we receive an HTTP 500 error and a developer error message that says:

Developer Error Message: No outgoing languages for cooperation: 17544
Developer Error Message: No outgoing languages for cooperation: 19273
Developer Error Message: No outgoing languages for cooperation: 19479

I think this is the solution adopted by USOS as suggested by @janinamincer-daszkiewicz to avoid to use UND (undefined) together with not-yet-defined, that they limited to the mutual approved IIAs in v6 (and I don't understand why).

@janinamincer-daszkiewicz seemed to be satisfied with this solution, but now that I understand what she meant and how they implement the solution, I think I can say that it's illegal and it infringes a basic rule:

You MUST be consistent. If you allow a specific requester to see a specific IIA, then this IIA MUST also be listed (to this requester) via the index endpoint. Similarly, if you leave a reference to an IIA when the requester is using any of the other APIs, then this requester MUST also be allowed to read this IIA

Indeed, the presence of a developer message means that the partner deliberately forbids us to read the IIAs that are listed in the Index, and this is not allowed by the rules.
I hope you realize that you encouraged to infringe old and commonly accepted rules just to avoid the use of the UND as language code and the not-yet-defined attribute for not mutually approved IIAs

[janinamincer-daszkiewicz]

These IIAs are not mutually approved, not even matched. They do not have languages filled in and University of Warsaw should not make them visible in the EWP network. Because they are visible but languages are missing the system does not allow to send them via the network but generates 500.

If they would be mutually approved but languages would be missing, they would have been sent with not-yet-defined and some language (not UND).

[umesh-qs]

These IIAs are not mutually approved, not even matched. They do not have languages filled in and University of Warsaw should not make them visible in the EWP network. Because they are visible but languages are missing the system does not allow to send them via the network but generates 500.

If they would be mutually approved but languages would be missing, they would have been sent with not-yet-defined and some language (not UND).

You are defining your own rules. If IIA ID is there in the index endpoint, you must not respond with 500 for IIA GET. If you think IIA is not appropriate to be shared then you must not put it in the index endpoint response. There is no confusion about it.

At this point I must ask, are you working for EWP project under DG EAC, or are you working for University of Warsaw implementation?

[umesh-qs]

These IIAs are not mutually approved, not even matched. They do not have languages filled in and University of Warsaw should not make them visible in the EWP network. Because they are visible but languages are missing the system does not allow to send them via the network but generates 500.
If they would be mutually approved but languages would be missing, they would have been sent with not-yet-defined and some language (not UND).

You are defining your own rules. If IIA ID is there in the index endpoint, you must not respond with 500 for IIA GET. If you think IIA is not appropriate to be shared then you must not put it in the index endpoint response. There is no confusion about it.

At this point I must ask, are you working for EWP project under DG EAC, or are you working for University of Warsaw implementation?

Probably you are putting hack in your system to mitigate the badly conceived API specification about deleted IIAs.

[Emkas]

I'll try to make it as simple as I can. USOS/MUCI provides software, that is being installed on HEIs. We do not own their data, we do not have access to their data. There were several points to get us where we are now. Several meetings, several new versions of software, one month of collecting the data about approved IIAs, so HEIs can switch to IIA 7.0.

What is happening here is that UW decides that they want all old agreements to be visible in EWP, even if they were approved only on paper. They bypass our scripts and our software and change the database data directly. Without looking on consequences. Now the only thing that we can do was: hide this behind their back (hide it on index) or be open about what is happening so that is someone will start making call for those IIAs both sides will see the problem and, hopefully, the problem will be fixed ASAP.

As a side note, to be fair, it was my call, @janinamincer-daszkiewicz was against it. I don't like to hide problems. We're catching them on stats portal and contacting HEIs, so they can fixed something that they have more than 6 months to do. This is all we can do, ask. As I said, we do not have access to HEIs data.

PS. This was months before UND and yet another shitstorm on github.
PS2. I don't know how can someone claim that this is a reference implementation. This is a closed code ;-)
PS3. Maybe you saw on Agreements form on HEI/DEMO installation that for about a year we have special filter called "EWP: missing languages". There is nothing I/we can do that some HEIs are ignoring all I/we told them on conferences, calls, multiple e-mails and on our issue tracker.
PS4. My understanding of EWP is that we want HEIs to exchange data electronically. I do whatever I can so that can be done as smooth and as fast as possible. I've got filling that this is not what some parties in this network wants. I think that if half of the energy that is put to wage wars on github, would be put to the implementation (or spec), we'd be in much better place right now.

[umesh-qs]

I'll try to make it as simple as I can. USOS/MUCI provides software, that is being installed on HEIs. We do not own their data, we do not have access to their data. There were several points to get us where we are now. Several meetings, several new versions of software, one month of collecting the data about approved IIAs, so HEIs can switch to IIA 7.0.

What is happening here is that UW decides that they want all old agreements to be visible in EWP, even if they were approved only on paper. They bypass our scripts and our software and change the database data directly. Without looking on consequences. Now the only thing that we can do was: hide this behind their back (hide it on index) or be open about what is happening so that is someone will start making call for those IIAs both sides will see the problem and, hopefully, the problem will be fixed ASAP.

As a side note, to be fair, it was my call, @janinamincer-daszkiewicz was against it. I don't like to hide problems. We're catching them on stats portal and contacting HEIs, so they can fixed something that they have more than 6 months to do. This is all we can do, ask. As I said, we do not have access to HEIs data.

PS. This was months before UND and yet another shitstorm on github. PS2. I don't know how can someone claim that this is a reference implementation. This is a closed code ;-) PS3. Maybe you saw on Agreements form on HEI/DEMO installation that for about a year we have special filter called "EWP: missing languages". There is nothing I/we can do that some HEIs are ignoring all I/we told them on conferences, calls, multiple e-mails and on our issue tracker. PS4. My understanding of EWP is that we want HEIs to exchange data electronically. I do whatever I can so that can be done as smooth and as fast as possible. I've got filling that this is not what some parties in this network wants. I think that if half of the energy that is put to wage wars on github, would be put to the implementation (or spec), we'd be in much better place right now.

Hi @Emkas As I understand from your response, some data in UW DB is bad/wrong. Even then, you must provide IIA details in the GET response. The partner will check and report about the wrong data.
By adding this 500 hack and Janina defending it, you are now legalizing the concept of draft IIAs

[umesh-qs]

I'll try to make it as simple as I can. USOS/MUCI provides software, that is being installed on HEIs. We do not own their data, we do not have access to their data. There were several points to get us where we are now. Several meetings, several new versions of software, one month of collecting the data about approved IIAs, so HEIs can switch to IIA 7.0.

What is happening here is that UW decides that they want all old agreements to be visible in EWP, even if they were approved only on paper. They bypass our scripts and our software and change the database data directly. Without looking on consequences. Now the only thing that we can do was: hide this behind their back (hide it on index) or be open about what is happening so that is someone will start making call for those IIAs both sides will see the problem and, hopefully, the problem will be fixed ASAP.

As a side note, to be fair, it was my call, @janinamincer-daszkiewicz was against it. I don't like to hide problems. We're catching them on stats portal and contacting HEIs, so they can fixed something that they have more than 6 months to do. This is all we can do, ask. As I said, we do not have access to HEIs data.

PS. This was months before UND and yet another shitstorm on github. PS2. I don't know how can someone claim that this is a reference implementation. This is a closed code ;-) PS3. Maybe you saw on Agreements form on HEI/DEMO installation that for about a year we have special filter called "EWP: missing languages". There is nothing I/we can do that some HEIs are ignoring all I/we told them on conferences, calls, multiple e-mails and on our issue tracker. PS4. My understanding of EWP is that we want HEIs to exchange data electronically. I do whatever I can so that can be done as smooth and as fast as possible. I've got filling that this is not what some parties in this network wants. I think that if half of the energy that is put to wage wars on github, would be put to the implementation (or spec), we'd be in much better place right now.

Better implementation will automatically from clear and better specs. And better specs will only come with open thought process. Not by limiting ourselves to knowledge of how few systems work and certainly not be quoting "Implementation is as per requirement from DG EAC".

[skishk]

we'd be in much better place right now

if we all respect the specification and not introduce own rules.
if the specification are changing just to be compliant for 1 system against all the rules and specification of Erasmus Program
The result could not be good... as we are seeing...

[skishk]

Because they are visible but languages are missing the system does not allow to send them via the network but generates 500.

Do you think is correct that?

[Emkas]

Hi @Emkas As I understand from your response, some data in UW DB is bad/wrong.

That's right. Maybe what is missing here is that this case is one-time problem. What I mean by that, is that for all new agreements this won't take place. This was a part of transition from v6 to v7.

Even then, you must provide IIA details in the GET response. The partner will check and report about the wrong data.

Well, the second part is the point of it. So that IRO's will contact each other, as their are doing already and say, something is wrong in your data, check it.

By adding this 500 hack and Janina defending it, you are now legalizing the concept of draft IIAs
That part about draft I don't understand, but I don't follow Github and concepts. I come when the specification is ready, since I assume this is mutual agreement of many parties and heads smarter than me :-)

About hack... Well, I'd call that this is an honest solution. I'd call a hack 500 without any information, and pretend that "someone is looking what is goin' on". As I know that some (rather smaller) parties are doing. But I don't what to be the sheriff on this network. I don't have time for this. For me EWP is less than 1% of our software which generate more than 50% of problems on our issue tracker.

[umesh-qs]

Hi @Emkas As I understand from your response, some data in UW DB is bad/wrong.

That's right. Maybe what is missing here is that this case is one-time problem. What I mean by that, is that for all new agreements this won't take place. This was a part of transition from v6 to v7.

Even then, you must provide IIA details in the GET response. The partner will check and report about the wrong data.

Well, the second part is the point of it. So that IRO's will contact each other, as their are doing already and say, something is wrong in your data, check it.

By adding this 500 hack and Janina defending it, you are now legalizing the concept of draft IIAs
That part about draft I don't understand, but I don't follow Github and concepts. I come when the specification is ready, since I assume this is mutual agreement of many parties and heads smarter than me :-)

About hack... Well, I'd call that this is an honest solution. I'd call a hack 500 without any information, and pretend that "someone is looking what is goin' on". As I know that some (rather smaller) parties are doing. But I don't what to be the sheriff on this network. I don't have time for this. For me EWP is less than 1% of our software which generate more than 50% of problems on our issue tracker.

A bit funny that you leave it to smarter parties to draft specification but still come up with your own interpretation and hack. I hope there is someone from your team who represent those parties.

You have assumed that every system will have implementation to show the message to the IROs that you pass along 500 response. 500 means its a server error which will is temporary and most likely get corrected on its own.
You are now trying to use it as your honest solution. But this is a hack that also opens up door for creating draft IIAs, share them and then un-share then from GET response and keep it in index response.

[skishk]

@Emkas no one is pointing on you 😊
we all just see different behaviour when we face incorrect implementation... and the solution leading just on one way... probably not the correct one in my opinion.

but as you said :

I don't have time for this. For me EWP is less than 1% of our software which generate more than 50% of problems on our issue tracker.

probably we all must do the same because the final result it is worst then the problem.

[demilatof]

I cannot add any more to what @umesh-qs and @skishk have said.
The HTTP 500 Error code indicates that the server encountered an unexpected condition that prevented it from fulfilling the request.
Using it, encouraged by @janinamincer-daszkiewicz , you aren't only infringing the EWP rule that an IIA exposed in the index must be returned, but you also make a bad use of the standard error code by adding an explanation that shows, if still necessary, that it is not an unexpected condition.
As @skishk said I don't address you, but the fact that the maintainers have drive providers to make bad choices.

As concern the reference implementation, I said "almost"; we can use the expression "golden implementation" or similar, but I think it would be very difficult to suppose something different, when we know that Janina is participating in USOS project (since 1999) and at the same time she is the maintainer of the EWP project.

[demilatof]

Hi @Emkas As I understand from your response, some data in UW DB is bad/wrong. Even then, you must provide IIA details in the GET response. The partner will check and report about the wrong data. By adding this 500 hack and Janina defending it, you are now legalizing the concept of draft IIAs

@umesh-qs I don't understand why we have discussed about the deletion and the possibility that an IIA appears again in the network.
Just hide it and return error 500 with a developer message "under review".

[umesh-qs]

Hi @Emkas As I understand from your response, some data in UW DB is bad/wrong. Even then, you must provide IIA details in the GET response. The partner will check and report about the wrong data. By adding this 500 hack and Janina defending it, you are now legalizing the concept of draft IIAs

@umesh-qs I don't understand why we have discussed about the deletion and the possibility that an IIA appears again in the network. Just hide it and return error 500 with a developer message "under review".

That is why I called it a hack in my other comments. And EWP consortia member giving green signal to it means we can now create new use cases with it.
What is absolutely disappointing that EWP consortia, does not see any urgency to put an end to this, by clearly stating that it is wrong to use 500 status code for such situations.

[demilatof]

These IIAs are not mutually approved, not even matched. They do not have languages filled in and University of Warsaw should not make them visible in the EWP network. Because they are visible but languages are missing the system does not allow to send them via the network but generates 500.

If they would be mutually approved but languages would be missing, they would have been sent with not-yet-defined and some language (not UND).

Your answer is dated 26 July.
Since then, MUCI (USOS) has stuffed the stats, being reported almost 600 times for http error 500:

https://stats.erasmuswithoutpaper.eu/monitoring/?group_by=&date_from=2024-07-26&server_message=+language&http_code=500&sort_by=-timestamp

MobilityOnLine only 8 times: https://stats.erasmuswithoutpaper.eu/monitoring/?group_by=&date_from=2024-07-26&server_message=+language&http_code=500&server_provider=mobility&sort_by=-timestamp

Your solution has been used only by MUCI (USOS), or you have imposed the solution used by MUCI (USOS), doesn't make a great difference.
There is no more to add.