Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spike(scoring): Investigate Post Scoring System Limitations and Gaming Vulnerabilities #36

Closed
Tracked by #88
teslashibe opened this issue Dec 16, 2024 · 12 comments · May be fixed by #96
Closed
Tracked by #88

spike(scoring): Investigate Post Scoring System Limitations and Gaming Vulnerabilities #36

teslashibe opened this issue Dec 16, 2024 · 12 comments · May be fixed by #96
Assignees

Comments

@teslashibe
Copy link
Contributor

teslashibe commented Dec 16, 2024

[SPIKE] Investigate Post Scoring System Limitations and Gaming Vulnerabilities

Problem Statement

The current scoring system (@post_scorer.py) has critical limitations in accurately measuring genuine engagement and content quality. The system's simplistic scoring metrics make it vulnerable to gaming and manipulation, potentially rewarding artificial rather than authentic social media engagement.

Current Issues

Engagement Metric Vulnerabilities

  • Simple normalization factors (100 likes, 50 replies, 75 retweets, 1000 views) are easily gameable
  • Log transformation doesn't sufficiently address viral outliers or detect artificial inflation
  • No distinction between quality of engagement sources (bot vs real users)
  • Engagement scoring treats all interaction types with equal validity

Content Quality Limitations

  • Binary scoring for media presence (0.5 for any media) ignores quality and relevance
  • Text length scoring (60-200 chars optimal) doesn't measure content value
  • No measurement of content originality or uniqueness
  • Limited content type differentiation (Photos, Videos, GIFs, URLs all treated equally)

Interaction Scoring Flaws

  • Conversation scoring (1.0 for any conversation) doesn't measure quality
  • Mention system (optimal at 1-2 mentions) can be gamed through spam
  • Thread detection is binary (0.5 for self-threads) without quality assessment
  • No measurement of authentic community engagement

Structural Issues

  • Fixed weight distribution (40% engagement, 30% content, 30% interaction) may not reflect true value
  • No temporal analysis of posting patterns or engagement growth
  • Lacks mechanisms to detect coordinated behavior between agents
  • Cannot identify or penalize artificial boost networks

Impact

  • Agents can achieve high scores through mechanical optimization rather than quality
  • Risk of reward distribution being skewed by gaming rather than merit
  • No differentiation between organic and artificial engagement
  • System may incentivize spam and low-quality content optimization

Evidence if Gaming Happening

  • Multiple agents showing identical engagement patterns
  • Suspicious rapid engagement growth on certain posts
  • Cross-promotion patterns between agent networks
  • Optimization around scoring thresholds rather than content quality

Questions to Address

  • How can we detect and measure authentic vs artificial engagement?
  • What metrics would better reflect genuine content value?
  • How can we incorporate engagement quality into scoring?
  • What mechanisms could prevent coordinated gaming?
  • How can we balance complexity vs gaming resistance?

Labels: scoring, security, investigation, high-priority

@teslashibe teslashibe changed the title spike(scoring): review current scoring from miner (posts.json) and understand and iterate on the current scoring model spike(scoring): Investigate Post Scoring System Limitations and Gaming Vulnerabilities Dec 27, 2024
@teslashibe
Copy link
Contributor Author

@ryssroad

This comment was marked as resolved.

@teslashibe
Copy link
Contributor Author

@ryssroad I created a new issue for this suggestion #51

We will cleanup this ticket and think about ways we can use evaluation of conversation in scoring

@teslashibe
Copy link
Contributor Author

teslashibe commented Dec 30, 2024

Verification that a user is an agent on X

X has a requirement that all agents on X are marked as an automated account (agent).

Read X's terms of service here: https://help.x.com/en/using-x/automated-account-labels

Therefore, we can use the X public API to verify that an X profile is indeed an automated account (agent).

This is the truncated response body that we can GET from X for X profile:

{
  "errors": [...],
  "data": {
    "user": {
      "result": {
        // Highlighted section
        "__typename": "User",
        "id": "VXNlcjoxODYyMTAyNjUzNDY5NDMzODU2",
        "rest_id": "1862102653469433856",
        "affiliates_highlighted_label": {
          "label": {
            "badge": {
              "url": "https://pbs.twimg.com/semantic_core_img/1428827730364096519/4ZXpTBhS?format=png&name=orig"
            },
            "description": "Automated", 
            "longDescription": {
              "text": "Automated by @CreatorBid",
              "entities": [
                {
                  "fromIndex": 13,
                  "toIndex": 24,
                  "ref": {
                    "type": "TimelineRichTextMention",
                    "screen_name": "CreatorBid",
                    "mention_results": {
                      "result": {
                        "__typename": "User",
                        "legacy": {
                          "screen_name": "CreatorBid"
                        },
                        "rest_id": "1747737231916240896"
                      }
                    }
                  }
                }
              ]
            },
            "userLabelType": "AutomatedLabel"
          }
        },
        // Rest of response truncated
        "is_blue_verified": true,
        "profile_image_shape": "Circle",
        "legacy": {...},
        ...
      }
    }
  }
}

We propose adding verification that an X account is automated as an immediate improvement to miner/agent registration and make this an immediate criteria not only for registration to the subnet but implicitly for scoring.

Here is a feature ticket for this improvement: #52

@teslashibe
Copy link
Contributor Author

Community feedback

My opinion: Skip likes, retweets, and comments until they can be verified. Focus on content from the AI agent instead. The more posts a miner has, the harder and costlier it becomes to fake engagement. Reward miners who frequently post, especially on diverse topics. For example, if a miner posts when BTC hits $100,000 and mentions it explicitly (not just saying it's bullish on BTC), it shows effort. This could require web searches at times. Rank miners higher when they comment on news articles and add something meaningful to the discussion, for instance.

@teslashibe
Copy link
Contributor Author

teslashibe commented Dec 30, 2024

[SPIKE] Comment: LLM-Based Tweet Quality Assessment

After investigating the current scoring vulnerabilities outlined in the spike ticket, we propose leveraging Subnet 19's LLM capabilities to create a more sophisticated content quality assessment system.

Key Insights

  • Basic engagement metrics are easily gamed and don't reflect true content value
  • Semantic analysis via LLM could provide deeper quality assessment
  • Integration with Subnet 19 offers natural pathway for implementation

Proposed Feature

See full feature ticket: [FEATURE] Integrate Subnet 19 LLM for Enhanced Tweet Quality Assessment
Link: #55

The feature proposes:

  1. Using LLM to score multiple quality dimensions (coherence, value, originality)
  2. Analyzing conversational depth and engagement quality
  3. Integrating with existing scoring while maintaining gaming resistance

Next Steps

  1. Review feature ticket proposal
  2. Technical feasibility assessment with Subnet 19 integration

/cc @hide-on-bush-x @theMultitude @grantdfoster @mudler @Luka-Loncar

Let's discuss the technical feasibility of this approach - this is low hanging fruit and we can easily spin up some notebooks to do an analysis here on the impact.

@teslashibe
Copy link
Contributor Author

teslashibe commented Dec 30, 2024

Linking to feature ticket for Subnet 19 integration bonus: #55

This feature will add a scoring bonus for SN59 agents that use Subnet 19 for inference, requiring:

  1. Integration with SN19 metrics API for usage verification:
GET "https://api.nineteen.com/v1/metrics/{nineteen-userid}"
  1. Implementation of scoring bonus (1.2x for example) for verified active users
  2. Caching and minimum thresholds for verification

Will help drive quality content generation and create stronger subnet synergies.

See full ticket for technical details and acceptance criteria.

cc @namoray for input on metrics API implementation

@teslashibe
Copy link
Contributor Author

Quick win updates and bug fixes here: #61

@ryssroad
Copy link

ryssroad commented Jan 4, 2025

Also, you can use post interval checking to determine manual posting, it is quite difficult to keep a strict schedule manually

@teslashibe
Copy link
Contributor Author

@ryssroad we will do an analysis here

@Luka-Loncar
Copy link

We should create follow up tasks on this Spike - syncing with the team on next grooming.

@teslashibe
Copy link
Contributor Author

This results into Scoring v4: #96

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants