Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experiment pixels #3669

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

experiment pixels #3669

wants to merge 6 commits into from

Conversation

SabrinaTardio
Copy link
Contributor

Task/Issue URL: https://app.asana.com/0/1204186595873227/1208775176602791/f
Tech Design URL: https://app.asana.com/0/1204186595873227/1208682592686299
CC:

Description: Implements PixelExperimentKit to provide Experiment metrics API to clients and fires pixel for preset experiment metrics

Steps to test this PR:

  1. Somewhere in the code base (maybe in NewTabPageViewModel) add this Experiment Feature Flag:
    public struct CredentialsSavingFlag: FeatureFlagExperimentDescribing {
    public init() {}

    public typealias CohortType = Cohort

    public var rawValue = "credentialSaving"

    public var source: FeatureFlagSource = .remoteReleasable(.subfeature(AutofillSubfeature.credentialsSaving))

    public enum Cohort: String, FlagCohort {
    case control
    case blue
    }
    }

  2. Assign the cohort somewhere (for example in NewTabPageViewModel initialiser)
    let cohort = Application.appDelegate.featureFlagger.getCohortIfEnabled(for: CredentialsSavingFlag())

  3. In AppURLs.swift change the PrivacyConfig URL to https://www.jsonblob.com/api/jsonBlob/1313198249494110208

  4. Run the app and check that the following pixel is sent: m_mac_experiment_enroll_credentialsSaving_blue (or control) with the expected parameters

  5. Perform 4 searches and and foreground the app 4 times check that only one experiment pixel is fired: m_ios_experiment_metrics_credentialsSaving_blue ["pixelSource": "browser-dmg", "metric": "search", "enrollmentDate": "2024-12-04", "value": "1", "conversionWindowDays": "0”]

Move the system date to the following day and perform 4 searches and 4 foregorunding, only 2 pixels should be fired:
m_ios_experiment_metrics_credentialsSaving_blue ["pixelSource": "browser-dmg", "metric": "search", "enrollmentDate": "2024-12-05", "value": "1", "conversionWindowDays": “1”]
m_ios_experiment_metrics_credentialsSaving_blue ["pixelSource": "browser-dmg", "metric": “ app_use", "enrollmentDate": "2024-12-05", "value": "1", "conversionWindowDays": “1”]

7 Try the same up to day 4

8 On day 5 do the same and in this case you should see multiple pixels:
👾[Unique By Name And Parameters-Fired] m_mac_experiment_metrics_credentialsSaving_control ["conversionWindowDays": "5", "value": "1", "enrollmentDate": "2024-12-04", "metric": "search", "pixelSource": "browser-dmg"]
👾[Unique By Name And Parameters-Fired] m_mac_experiment_metrics_credentialsSaving_control ["pixelSource": "browser-dmg", "conversionWindowDays": "5-7", "enrollmentDate": "2024-12-04", "value": "1", "metric": "search"]
👾[Unique By Name And Parameters-Fired] m_mac_experiment_metrics_credentialsSaving_control ["pixelSource": "browser-dmg", "value": "1", "conversionWindowDays": "5", "metric": "app_use", "enrollmentDate": "2024-12-04"]
👾[Unique By Name And Parameters-Fired] m_mac_experiment_metrics_credentialsSaving_control ["metric": "app_use", "conversionWindowDays": "5-7", "pixelSource": "browser-dmg", "enrollmentDate": "2024-12-04", "value": "1"]
👾[Unique By Name And Parameters-Fired] m_mac_experiment_metrics_credentialsSaving_control ["value": "4", "pixelSource": "browser-dmg", "conversionWindowDays": "5-7", "metric": "search", "enrollmentDate": "2024-12-04"]
👾[Unique By Name And Parameters-Fired] m_mac_experiment_metrics_credentialsSaving_control ["value": "4", "enrollmentDate": "2024-12-04", "metric": "app_use", "pixelSource": "browser-dmg", "conversionWindowDays": "5-7”]

Note: I recommend Filter by Unique By Name And Parameters-Fired

Definition of Done (Internal Only):

Copy Testing:

  • Use of correct apostrophes in new copy, ie rather than '

Orientation Testing:

  • Portrait
  • Landscape

Device Testing:

  • iPhone SE (1st Gen)
  • iPhone 8
  • iPhone X
  • iPhone 14 Pro
  • iPad

OS Testing:

  • iOS 15
  • iOS 16
  • iOS 17

Theme Testing:

  • Light theme
  • Dark theme

Internal references:

Software Engineering Expectations
Technical Design Template

@SabrinaTardio SabrinaTardio marked this pull request as ready for review December 4, 2024 12:50
@SabrinaTardio SabrinaTardio changed the title Sabrina/experiment pixels experiment pixels Dec 4, 2024
Copy link
Contributor

@jaceklyp jaceklyp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants