Skip to content

Graph node dev mode #5982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: krishna/file-link-resolver
Choose a base branch
from

Conversation

incrypto32
Copy link
Member

@incrypto32 incrypto32 commented May 1, 2025

  • Uses File based link resolver to run subgraphs from file system
  • Listen for file system events to remove and redeploy subgraphs during development
  • Uses pg_temp to spin up temporary databases in build dirs of subgraphs

Follow up for #5977

@incrypto32 incrypto32 force-pushed the krishna/graph-dev branch from 2783041 to a414de1 Compare May 1, 2025 12:59
@incrypto32 incrypto32 changed the base branch from master to krishna/file-link-resolver May 1, 2025 13:02
@incrypto32 incrypto32 force-pushed the krishna/graph-dev branch from a414de1 to ff19f99 Compare May 1, 2025 13:55
@incrypto32 incrypto32 self-assigned this May 1, 2025
@lutter lutter self-requested a review May 1, 2025 15:54
@incrypto32 incrypto32 force-pushed the krishna/file-link-resolver branch from 3ef16d5 to 23e91f1 Compare May 6, 2025 12:59
@incrypto32 incrypto32 force-pushed the krishna/graph-dev branch from ff19f99 to e5a9e23 Compare May 6, 2025 13:02
Copy link
Collaborator

@lutter lutter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks great! There are a few improvements that would streamline the code, but this is a really good addition

file_link_resolver
} else {
Arc::new(IpfsResolver::new(ipfs_client, env_vars.cheap_clone()))
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

match would be more idiomatic here. Alternatively, have run take a Arc<dyn LinkResolver> and leave it up to each caller what they want to pass in


// Once we receive an event, wait for a short period of time to allow for multiple events to be received
// This is because running graph build writes multiple files at once
// Which triggers multiple events, we only need to react to it once
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be more robust to have graph-cli touch a file as the very last thing it does and only watch that. That would also make exclusions etc. unnecessary. For now though, this is fine.

// If no exclusions, process all events
if exclusion_set.is_empty() {
return true;
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't needed since we'll just skip the for loop when there are no exclusions

.await;
});
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this a little odd to have the optional dev_ctx passed into run. Another approach would be to have a new function start that does most of what run does now except for the last line graph::futures03::future::pending::<()>().await so that graph-node's run function would call start(..) and then wait on that pending future whereas gnd's run would call start and then block on watch_subgraph_updates

.with_initdb_param("-E", "UTF8")
.with_initdb_param("--locale", "C")
.start_async()
.await;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be a nice enhancement if it was possible to pass a postgres URL in as an option, so that when you already have PG setup you can just use that, and use pgtemp if the user doesn't pass a URL in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants