Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate tracker build + tests w/ Vite, Turborepo, & Playwright #132

Merged
merged 38 commits into from
Dec 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
1e12a25
First start at turborepo
benvinegar Dec 19, 2024
b4e971c
Remove temp turbo files, fix public dir
benvinegar Dec 19, 2024
f63a3fb
Fix gitignore
benvinegar Dec 19, 2024
73e77a5
Add tracker subrepo
benvinegar Dec 20, 2024
76a2271
Move typescript to root package
benvinegar Dec 20, 2024
3010023
Move prettier to root project
benvinegar Dec 20, 2024
8cf9850
Empty tsconfig
benvinegar Dec 20, 2024
68c99b4
Copy dist/tracker.js to server /public dir
benvinegar Dec 20, 2024
2dcbe87
Fix tests, precommit hooks
benvinegar Dec 20, 2024
c758112
Add deploy task
benvinegar Dec 20, 2024
ddedb9d
Run single turbo run command on precommit
benvinegar Dec 20, 2024
e11b43c
Add playwright tests for tracker script
benvinegar Dec 20, 2024
63538d3
Add test to verify request to /collect
benvinegar Dec 20, 2024
d9cf3de
Remove example tests
benvinegar Dec 20, 2024
bb7872e
Test asserts query string is good
benvinegar Dec 20, 2024
ffbd8ec
Remove tmp files
benvinegar Dec 20, 2024
107002d
Attempt to fix GH actions
benvinegar Dec 20, 2024
b45b635
GH action fix attempt #2
benvinegar Dec 20, 2024
29ec617
Add wrangler to root package.json
benvinegar Dec 20, 2024
1303d6f
Update package-lock
benvinegar Dec 20, 2024
2acc062
Add "preview" turbo command, edit README w/ new turbo commands
benvinegar Dec 20, 2024
2567fe0
Convert tracker to typescript
benvinegar Dec 20, 2024
f3819f5
Get tracker tests into folders (get ready for multiple tests)
benvinegar Dec 21, 2024
a651bab
Rename tracker.js to reporter.js
benvinegar Dec 27, 2024
efc8d9c
Add missing /reporter dir
benvinegar Dec 27, 2024
9123caa
Add data-site-id initialization method
benvinegar Dec 27, 2024
e7f13d7
Update README for simplified initialization method
benvinegar Dec 27, 2024
dafb7ba
Rename workspaces to packages
benvinegar Dec 27, 2024
07252ca
Rename test file to better mirror 01_legacyInit
benvinegar Dec 27, 2024
50a9011
Add eslint to reporter package
benvinegar Dec 27, 2024
1754cf2
Typecheck needs to build first
benvinegar Dec 28, 2024
df018d7
Rollback newly introduced API methods (unnecessary)
benvinegar Dec 28, 2024
f551437
Update typescript
benvinegar Dec 28, 2024
6d8c212
Same typescript version in both packages
benvinegar Dec 28, 2024
7bb6b16
Update package lock
benvinegar Dec 28, 2024
a95865d
Rename reporter back to tracker
benvinegar Dec 28, 2024
f9cd9e1
Update eslint typescript tooling to suppress warnings
benvinegar Dec 28, 2024
14f1bb0
Update turbo dependencies
benvinegar Dec 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/cd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
with:
node-version-file: "package.json"
- run: npm install
- run: npm run build
- run: npx turbo build
- uses: cloudflare/wrangler-action@v3
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
Expand Down
5 changes: 1 addition & 4 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,7 @@ jobs:
with:
node-version-file: "package.json"
- run: npm install
- run: npm run lint
- run: npm run build
- run: npm run typecheck
- run: npm run test-ci
- run: npx turbo lint build typecheck test-ci --concurrency=1
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
env:
Expand Down
12 changes: 7 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
node_modules

/.cache
/.wrangler
/build
/public/build
.cache
.wrangler
build
dist
public/build
.env

.dev.vars

coverage
coverage
.turbo
4 changes: 1 addition & 3 deletions .husky/pre-commit
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@ echo "$FILES" | xargs ./node_modules/.bin/prettier --ignore-unknown --write
# Add back the modified/prettified files to staging
echo "$FILES" | xargs git add

npm test
npm run lint
npm run typecheck
npx turbo test lint typecheck --concurrency=1

exit 0
52 changes: 21 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,52 +22,42 @@ If you don't have one already, [create a Cloudflare account here](https://dash.c
1. Enable [Cloudflare Analytics Engine beta](https://developers.cloudflare.com/analytics/analytics-engine/get-started/) for your account ([screenshot](https://github.com/benvinegar/counterscale/assets/4562878/ad1b5712-2344-4489-a684-685b876635d1))
1. If this is your first time using Workers, you have to create a Worker before you can enable the Analytics Engine. Navigate to Workers & Pages > Overview, click the "Create Worker" button ([screenshot](./docs/create-worker.png)) to create a "Hello World" worker (it doesn't matter what you name this Worker as you can delete it later).
1. Create a [Cloudflare API token](https://developers.cloudflare.com/fundamentals/api/get-started/create-token/). This token needs `Account.Account Analytics` permissions at a minimum ([screenshot](./docs/api-token.png)).
* _WARNING: Keep this window open or copy your API token somewhere safe (e.g. a password manager), because if you close this window you will not be able to access this API token again and have to start over._
- _WARNING: Keep this window open or copy your API token somewhere safe (e.g. a password manager), because if you close this window you will not be able to access this API token again and have to start over._

### Deploy Counterscale

1. Download the [latest Counterscale release](https://github.com/benvinegar/counterscale/releases/latest) (or clone the repository) and extract the source files to a folder.
1. With your terminal, navigate to the folder containing the source files.
1. Run `npm install`
1. Run `npx wrangler pages project create counterscale` and create a new Pages project.
1. You will be prompted to enter the "production branch name". Just use the default provided (e.g. "main" or "production").
* _NOTE: If this is your first time invoking `wrangler` on the terminal, you will be prompted to sign into your Cloudflare account._
1. You will be prompted to enter the "production branch name". Just use the default provided.
- _NOTE: If this is your first time invoking `wrangler` on the terminal, you will be prompted to sign into your Cloudflare account._
1. Run `npx wrangler pages secret put CF_BEARER_TOKEN` → when prompted, paste the API token you created
1. Run `npx wrangler pages secret put CF_ACCOUNT_ID` → when prompted, paste your Cloudflare Account ID
* Find your account ID by visiting Workers and Pages > Overview. It is displayed on the right hand side of the screen.
1. Run `npm run deploy` – this will do several things:
- Find your account ID by visiting Workers and Pages > Overview. It is displayed on the right hand side of the screen.
1. Run `npx turbo deploy` – this will do several things:
1. Create a new Analytics Engine dataset, called `metricsDataset`
1. Deploy the site and give you the deployment URL.
1. The site should now be deployed. Visit `https://{subdomain-emitted-from-npm-run-deploy}.pages.dev`.
* NOTE: _It may take take a few minutes before the subdomain becomes live._
- NOTE: _It may take take a few minutes before the subdomain becomes live._

### Install the Tracker Script on Your Website(s)
### Install the Tracking Script on Your Website(s)

When Counterscale is deployed, it makes `tracker.js` available at the URL you deployed to:

```
https://{subdomain-emitted-from-npm-run-deploy}.pages.dev/tracker.js
```

To start tracking website traffic on your web property, copy/paste the following snippet into your website HTML:
To start reporting website traffic from your web property, copy/paste the following snippet into your website HTML:

```html
<script>
(function () {
window.counterscale = {
q: [["set", "siteId", "your-unique-site-id"], ["trackPageview"]],
};
})();
</script>
<script
id="counterscale-script"
<script
src="https://{subdomain-emitted-from-npm-run-deploy}.pages.dev/tracker.js"
defer
></script>
```

Be sure to replace `your-unique-site-id` with a unique string/slug representing your web property. Use a unique site ID for each property you place the tracking script on.

## Troubleshooting

If the website is not immediately available (e.g. "Secure Connection Failed"), it could be because Cloudflare has not yet activated your subdomain (yoursubdomain.workers.dev). This process can take a minute; you can check in on the progress by visiting the newly created worker in your Cloudflare dashboard (Workers & Pages → counterscale).
Expand All @@ -80,16 +70,16 @@ The deployment URL can always be changed to go behind a custom domain you own. [

### Config

To get started, in the project root, copy `.dev.vars.example` to `.dev.vars`.
To get started, in the `packages/server` folder, copy `.dev.vars.example` to `.dev.vars`.

Open `.dev.vars` and enter the same values for `CF_BEARER_TOKEN` and `CF_ACCOUNT_ID` you used earlier.

### Running the Server

Counterscale is built on Remix and Cloudflare Workers. In development, you have two options:

1. `npm run dev` → This runs the Vite development server in Node.js. This server will automatically rebuild files when you change them, but it does not best reflect Cloudflare's serverless platform.
2. `npm run preview` → This runs Cloudflare's Miniflare server with a build of the Remix files. This closer matches the deployment environment, but does not (yet) automatically rebuild your app.
1. `npx turbo dev` → This runs the Vite development server in Node.js. This server will automatically rebuild files when you change them, but it does not best reflect Cloudflare's serverless platform.
2. `npx turbo preview` → This runs Cloudflare's Miniflare server with a build of the Remix files. This closer matches the deployment environment, but does not (yet) automatically rebuild your app.

## Notes

Expand All @@ -99,8 +89,8 @@ There is only one "database": the Cloudflare Analytics Engine dataset, which is

Right now there is no local "test" database. This means in local development:

- Writes will no-op (no hits will be recorded)
- Reads will be read from the production Analaytics Engine dataset (local development shows production data)
- Writes will no-op (no hits will be recorded)
- Reads will be read from the production Analaytics Engine dataset (local development shows production data)

### Sampling

Expand All @@ -116,10 +106,10 @@ The primary goal of Counterscale is to be super easy to self-host and maintain.

To achieve that:

- There should be no application state outside of CF Analytics Engine
- e.g. no additional relational database like MySQL, PostgreSQL, etc.
- That means no `users` table, no `sites` table, etc.
- This also means retention will be limited by what CF Analytics Engine provides. While it could be possible to stand up a "hit counter" for long-lived data (e.g. years), that would mean another database, which we will not pursue.
- We prioritize backwards compatibility
- New `metricsDataset` columns can be added, but old columns cannot be removed or renamed (they can however, be "forgotten").
- That also means it's okay if a feature only works during a period where the data is active.
- There should be no application state outside of CF Analytics Engine
- e.g. no additional relational database like MySQL, PostgreSQL, etc.
- That means no `users` table, no `sites` table, etc.
- This also means retention will be limited by what CF Analytics Engine provides. While it could be possible to stand up a "hit counter" for long-lived data (e.g. years), that would mean another database, which we will not pursue.
- We prioritize backwards compatibility
- New `metricsDataset` columns can be added, but old columns cannot be removed or renamed (they can however, be "forgotten").
- That also means it's okay if a feature only works during a period where the data is active.
Loading
Loading