🔷 Epic: Replace Indexer

[Tguntenaar]

Issue

An indexer is essential in blockchain development as it preprocesses, organizes, and caches blockchain data, enabling efficient queries, real-time updates, and a seamless user experience that raw blockchain nodes cannot provide.

The NEAR Query API and indexer are going out of service on December 9, and the DevHub portal depends on them. To ensure uninterrupted functionality, we must replace both the indexer and the API.

Current API Usage

The API is currently used in the following places across multiple portals, including the DevHub Portal, Events Committee, Infrastructure Committee, and now also the Templar Portal:

• The proposal functionality is shared across all portals.
• The RFP functionality is specific to the Templar Portal and Infrastructure Committee.
• The Events Committee and Infrastructure Committee utilize the API for categories.

1. Proposal Dashboard
2. Proposal Page:
   • Fetch linked proposals.
   • Fetch linked RFPs.
   • Fetch timelines.
3. RFP Dashboard
4. RFP Page:
   • Fetch linked proposals.
   • Fetch timelines.
5. Dropdown Components:
   • linkedProposalsDropdown
   • linkedRfpDropdown
6. Tests:
   • Mocked API for unit and integration testing.
7. Markdown Editor:
   • Fetch getSuggestedProposals.
8. Subsequent API Calls:
   • Fetch detailed transaction data from the NEAR blockchain.
   • Use transaction data to determine the specific version of terms and conditions accepted with a proposal.
9. Posts Feeds:
   • Retrieve post and comment data.
   • Used on the now-archived feed page.
   • Also used in community pages, as well as the discussions and announcements feeds.
10. BlogV1 Addon

What Has Been Done

The focus so far has been on creating the indexer and deploying the API's. Now, we need to replace the API in the front end without breaking any features.

Next Steps:

1. Replace any part of the front end that uses the NEAR Query API, or sunset those parts. Replace any part of the front end that uses the NEAR Query API #968
2. Prioritize the proposals and RFP functionality while sunsetting: Replace any part of the front end that uses the NEAR Query API #968

• The BlogV1 addon.
• The now-archived post feeds.

3. Rewrite the front-end tests, as they currently mock the old API. Rewrite Front-End Tests to mock the New API #987

Future work

Since we have a hard deadline on December 9th I want to make sure we finish the most critical parts of this issue first.
Some feature are more criticial than others, the proposal/rfp dashboard, the creation flows are at the top so I'm putting these features of to fix at the end:

1. Index community announcement and discussions Sunset announcements and discussions #988
2. Rebuild the relationship management between Linked Rfp and Linked Proposals flows Add relationship management between Rfp and Proposals #989
3. Write tests for the new API, this is good to have for further development once it is production Write Tests for the New API #986
4. Fix the discussions test #991

[ori-near]

FYI: Thomas confirmed the QueryAPI is decommissioning on December 9

[ori-near]

@Tguntenaar please add more info to the remaining tickets & update the target deadline.

[Tguntenaar]

The first two subissues are done. The first one is in review. I will finish issue 968 this Firday the 8th.

[ori-near]

@Tguntenaar – Can you please update this ticket with the latest status? We moved into Sprint 3 because we still see things pending. But given that we've already pushed the deadline twice on this, we'd like to really see a path to resolution.

[Tguntenaar]

@ori-near I had written my update on the subissue, my bad! 968. The PR 982 resolves sub issue (3) and resolves this issue. Some of the tests fail because we mock answers from the old query endpoint.

Once those tests are resolved it is ready to merge.

[Tguntenaar]

I wanted to provide an update regarding the previous deadline estimation. While unblocking Templar and working on the RFPs for the Infrastructure Committee, we discovered there’s significantly more work involved than initially anticipated.

Given the complexity, writing the functionality and accompanying tests is going to take longer than expected. I’ll need more hands on this @Megha-Dev-19 @petersalomonsen

[ori-near]

Thank you @Tguntenaar for updating the scope of this epic ticket!

[Tguntenaar]

Current Status & Next Steps:

• Issues #968 & #987

  • The PR is reviewed and approved by Peter.
  • The remaining step is replacing the RPC in some tests, which will close these issues.
  • ETA: Tomorrow or Sunday

• #990 (Missing snapshot)

  • Megha reported this issue.
  • ETA: ~3 days.

• #989 (Linking RFP and proposals)

  • ETA: ~3 days.

• #988 (Sunsetting parts)

  • Created this as a separate issue since it needs a discussion with Vlad and the team to finalize what will be sunsetted.
    ETA: TBD, dependent on the discussion outcome.

• #986 (Biggest task: comprehensive testing)

  • I estimate this to be the largest task, but testing is being partially addressed in earlier PRs alongside feature changes.
  • This has the lowest priority right now, as I'm prioritizing snapshot history for RFPs and proposals and their linking.

Let me know if this prioritization aligns with your expectations or if you’d like me to adjust and focus differently.

[Tguntenaar]

I've updated this PR to intentionally skip the failing test due to RPC timeouts. I created a bug issue #991 to track the problem and temporarily disabled the test to unblock the pipeline. A note referencing the issue has been added in the source code. The impact of losing test coverage is minimal, as we are considering removing communities from the platform entirely