Gathering reposts of a 'root' post for the timestamp appears to be excessively difficult #3305
Unanswered
jsale
asked this question in
Clarifications and "Why?"
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
[UPDATE: More evidence of my noobishness, I only now found this page:
https://docs.bsky.app/docs/advanced-guides/timestamps
Why didn't any of the AI models say anything like what is said on this page?? Anyhow, this still doesn't necessarily solve my problem, and my post may still be of interest to some. I would still be grateful for any advice and suggestions]
This is my first post as a Bluesky developer. First, I want to say a huge THANKS for atproto. Up until Twitter was taken over and developers got stuck with exorbitant pricing, I did a lot of scraping and visualizing of Twitter data, mainly trying to find ways to visualize user behaviors in order to identify bots. I have attached a couple of examples of visualizing the 'life of a tweet' as examples. I am trying to do this with Bluesky now. After several attempts and interacting with a few of the public AI models, it appears they think that the only way to gather all reposts of an original post is to get all of the users who reposted, using the getRepostedBy function, and then go through them one by one and search their timelines for the reposted post. Not only can this be alot of API calls and I want to respect rate limits because I am also scraping the firehose and scraping user timelines for other reasons so this would take a very long time to get the 'life of a post' of any significant size, but it turns out that the createdAt timestamp of the original post returned from scraping the firehose and the createdAt timestamp from one of the user's timelines who reposted the post are identical. For example, here's the original post I'm looking at, scraped from the firehose, reposted ~600 times:
{"text": "Wow, Trump is still insulting Ron Desantis at his club calling him \u201cRon Desanctimonious.\u201d I thought they patched things up and they were friends now? He even considered him for SecDef. What happened?", "created_at": "2024-12-22T01:08:59.653Z", "author": "ronfilipkowski.bsky.social", "uri": "at://did:plc:t6ubj2wlhc34awzcymh3qpur/app.bsky.feed.post/3ldua5a3hwc2k", "has_images": false, "reply_to": null}
Here is one of the reposts from scraping the user's timeline:
{"uri": "at://did:plc:t6ubj2wlhc34awzcymh3qpur/app.bsky.feed.post/3ldua5a3hwc2k", "text": "Wow, Trump is still insulting Ron Desantis at his club calling him “Ron Desanctimonious.” I thought they patched things up and they were friends now? He even considered him for SecDef. What happened?", "created_at": "2024-12-22T01:08:59.653Z", "langs": ["en"], "tags": null, "embed": {"cid": "bafkreifta54p3j6fnwb4myzqevlmdptiw7igq4mzl4ca3553imjizvki2y", "playlist": "https://video.bsky.app/watch/did%3Aplc%3At6ubj2wlhc34awzcymh3qpur/bafkreifta54p3j6fnwb4myzqevlmdptiw7igq4mzl4ca3553imjizvki2y/playlist.m3u8", "alt": null, "aspect_ratio": {"height": 1088, "width": 886, "py_type": "app.bsky.embed.defs#aspectRatio"}, "thumbnail": "https://video.bsky.app/watch/did%3Aplc%3At6ubj2wlhc34awzcymh3qpur/bafkreifta54p3j6fnwb4myzqevlmdptiw7igq4mzl4ca3553imjizvki2y/thumbnail.jpg", "py_type": "app.bsky.embed.video#view"}, "reply": null, "facets": null, "type": null}
Any suggestions? I'm definitely a noob.
Beta Was this translation helpful? Give feedback.
All reactions