forked from webrecorder/browsertrix-behaviors
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* instagram behavior: - add new status messages - open first post in new window, inject behaviors with autofetch into new window, close when done - add ci for instagram behavior test with profile - ensure all tests use behaviors.js from current branch added as volume - add logged in twitter behavior with profile - load profiles from config specified in environment (loading from DigitalOcean) - use browsertrix image as specified in $BROWSERTRIX_IMAGE env var loaded from test.env - Update README.md with badges for new tests
- Loading branch information
Showing
16 changed files
with
305 additions
and
72 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| name: "Instagram Behavior (Logged In)" | ||
|
|
||
| on: | ||
| push: | ||
| pull_request: | ||
| schedule: | ||
| - cron: '15 1 * * *' | ||
|
|
||
| jobs: | ||
| instagram_test: | ||
| environment: DigitalOcean | ||
| runs-on: ubuntu-latest | ||
|
|
||
| strategy: | ||
| matrix: | ||
| node-version: [14.x] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v1 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
|
|
||
| - uses: prewk/s3-cp-action@master | ||
| env: | ||
| AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| AWS_S3_ENDPOINT: ${{ secrets.AWS_S3_ENDPOINT }} | ||
| SOURCE: 's3://dh-preserve/test-profiles/instagram/instagram.tar.gz' | ||
| DEST: './profile.tar.gz' | ||
|
|
||
| - name: install requirements | ||
| run: yarn install | ||
|
|
||
| - uses: c-py/action-dotenv-to-setenv@v2 | ||
| with: | ||
| env-file: test/test.env | ||
|
|
||
| - name: get browsertrix image | ||
| run: docker pull $BROWSERTRIX_IMAGE | ||
|
|
||
| - name: run crawl | ||
| run: docker run -v $PWD/dist/behaviors.js:/app/node_modules/browsertrix-behaviors/dist/behaviors.js -v $PWD/profile.tar.gz:/profile.tar.gz $BROWSERTRIX_IMAGE crawl --url https://instagram.com/capturesample --limit 1 --logging behaviors --profile /profile.tar.gz > ./log | ||
|
|
||
| - name: cat log | ||
| run: cat ./log | ||
|
|
||
| - name: compare crawl log to expected | ||
| run: grep '"state"' ./log | diff - ./test/expected-instagram.log | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| name: "Twitter Behavior (Logged In)" | ||
|
|
||
| on: | ||
| push: | ||
| pull_request: | ||
| schedule: | ||
| - cron: '14 1 * * *' | ||
|
|
||
| jobs: | ||
| twitter_logged_in_test: | ||
| environment: DigitalOcean | ||
| runs-on: ubuntu-latest | ||
|
|
||
| strategy: | ||
| matrix: | ||
| node-version: [14.x] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v1 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
|
|
||
| - uses: prewk/s3-cp-action@master | ||
| env: | ||
| AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| AWS_S3_ENDPOINT: ${{ secrets.AWS_S3_ENDPOINT }} | ||
| SOURCE: 's3://dh-preserve/test-profiles/twitter/twitter.tar.gz' | ||
| DEST: './profile.tar.gz' | ||
|
|
||
| - name: install requirements | ||
| run: yarn install | ||
|
|
||
| - uses: c-py/action-dotenv-to-setenv@v2 | ||
| with: | ||
| env-file: test/test.env | ||
|
|
||
| - name: get browsertrix image | ||
| run: docker pull $BROWSERTRIX_IMAGE | ||
|
|
||
| - name: run crawl | ||
| run: docker run -v $PWD/dist/behaviors.js:/app/node_modules/browsertrix-behaviors/dist/behaviors.js -v $PWD/profile.tar.gz:/profile.tar.gz $BROWSERTRIX_IMAGE crawl --url https://twitter.com/CaptureSample --limit 1 --logging behaviors --profile /profile.tar.gz > ./log | ||
|
|
||
| - name: cat log | ||
| run: cat ./log | ||
|
|
||
| - name: compare crawl log to expected | ||
| run: grep '"state"' ./log | diff - ./test/expected-twitter-logged-in.log | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.