Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming post stops after 30 mins #16

Open
cookiejul opened this issue Sep 11, 2024 · 6 comments
Open

Streaming post stops after 30 mins #16

cookiejul opened this issue Sep 11, 2024 · 6 comments

Comments

@cookiejul
Copy link

How can I resolve this issue? Seems like a similar issue with #10

@thecsw
Copy link
Owner

thecsw commented Sep 11, 2024

Are you seeing the issue for any streaming or under specific conditions? Are there any logs or tracebacks available for when the issue starts occurring?

@cookiejul
Copy link
Author

cookiejul commented Sep 11, 2024

No logs and no errors. Here's my code. Quite straightforward actually

r, _ := mira.Init(mira.ReadCredsFromFile("login.conf"))
c, _ := r.Subreddit("news").StreamSubmissions()
for {
	post := <-c

	if post.GetUrl() != "" {
		fmt.Printf("[%s] Creating post for: %s with links: %s", post.GetSubreddit(), post.GetTitle(), post.GetUrl())
		sendPostToDb(post)
	}
}

It just stops streaming after 30 mins. Do you have this issue? It's quite hard to debug because it doesn't throw any errors at all

@cookiejul
Copy link
Author

Hi @thecsw, sorry to have to bother you again. Do you happen to know the reason why it stops streaming? Were you able to replicate the issue on your side?

@thecsw
Copy link
Owner

thecsw commented Sep 14, 2024

Hello, @cookiejul! I'm afraid I don't have the time nowadays to look more in-depth into it, as I actually no longer even use reddit nor have an easy access to any accounts or APIs. This might need to be an issue that sees a solution from the community or wait until I can get to it.

I've been running a startup for a couple of years using mira's streaming and it was always working fine. I have multiple thoughts on how I would start triaging it:

  • Did Reddit update and tighten their API limits? With all the AI data collections, they've been on the news quite a bunch, especially with restricting developers from using Reddit API in any commercial sense.
  • Are you finding this issue with all the types of streaming? For me mensions, submissions, comments, replies, etc.? If for all, then it's some more global issue, otherwise, could be a specific stream issue.
  • Add additional logging (I haven't gotten to it) to debug and see whether we actually make a request and whether it silently fails
  • Do we make a request at all or is it some logic issue? I know I've implement a pointer implementation as recommended by Reddit circa 2019, but has that changed? Is the logic no longer holding true on how we stream?

Hopefully those points above introduce some ways one can go figuring it out and/or replicating the issue. Thanks

@cookiejul
Copy link
Author

Sounds good. I'll look into it and see if I can find out the root cause of this issue. If you don't mind sharing, what is your startup doing? Would love to check it out

@thecsw
Copy link
Owner

thecsw commented Sep 14, 2024

Long time ago, no longer active, it was a trading floor for meme subreddits, where one could buy and sell memes—there is a history page on https://sandyuraz.com/projects/memeinvestor_bot/

Thanks for looking into the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants