Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sanitise Data leaving Travis #20

Open
matt-oc opened this issue Apr 4, 2016 · 10 comments
Open

Sanitise Data leaving Travis #20

matt-oc opened this issue Apr 4, 2016 · 10 comments

Comments

@matt-oc
Copy link
Member

matt-oc commented Apr 4, 2016

The data leaving the Travis module (and npm and github) needs to be decided upon before it leaves. The problem is we have alot of data for each module and the data for each query may be different. Below are three example dumps from the Travis module:

Searching for Fuge:

{
  "name": "fuge",
  "url": "https:\/\/travis-ci.org\/apparatus\/fuge",
  "id": 6206563,
  "slug": "apparatus\/fuge",
  "active": true,
  "description": "",
  "last_build_id": 120434834,
  "last_build_number": "128",
  "last_build_state": "passed",
  "last_build_duration": 286,
  "last_build_language": null,
  "last_build_started_at": "2016-04-03T14:47:51Z",
  "last_build_finished_at": "2016-04-03T14:50:32Z",
  "github_language": "JavaScript",
  "sudo": false,
  "language": "node_js",
  "node_js": [
    0.1,
    0.12,
    4
  ],
  ".result": "configured",
  "group": "stable",
  "dist": "precise"
}

Lodash:

{
  "name": "lodash",
  "url": "https:\/\/travis-ci.org\/lodash\/lodash",
  "id": 48500,
  "slug": "lodash\/lodash",
  "active": true,
  "description": "A modern JavaScript utility library delivering modularity, performance, & extras.",
  "last_build_id": 120653169,
  "last_build_number": "4380",
  "last_build_state": "started",
  "last_build_duration": null,
  "last_build_language": null,
  "last_build_started_at": "2016-04-04T15:48:33Z",
  "last_build_finished_at": null,
  "github_language": "JavaScript",
  "language": "node_js",
  "sudo": false,
  "node_js": [
    "5"
  ],
  "env": [
    "",
    "BIN=\"phantomjs\"",
    "ISTANBUL=true",
    "SAUCE_LABS=true"
  ],
  "matrix": {
    "include": [
      {
        "node_js": "0.10",
        "env": null,
        "dist": "precise"
      },
      {
        "node_js": "0.12",
        "env": null,
        "dist": "precise"
      },
      {
        "node_js": "4",
        "env": null,
        "dist": "precise"
      }
    ]
  },
  "git": {
    "depth": 10
  },
  "branches": {
    "only": [
      "master"
    ]
  },
  "notifications": {
    "webhooks": {
      "urls": [
        "https:\/\/webhooks.gitter.im\/e\/4aab6358b0e9aed0b628"
      ],
      "on_success": "change",
      "on_failure": "always"
    }
  },
  "before_install": [
    "nvm use $TRAVIS_NODE_VERSION",
    "npm set loglevel error",
    "npm set progress false",
    "npm i -g npm@\"^2.0.0\"",
    "PATTERN[0]=\"|\\s*if\\s*\\(isHostObject\\b[\\s\\S]+?\\}(?=\\n)|\"\nPATTERN[1]=\"|\\s*if\\s*\\(enumerate\\b[\\s\\S]+?\\};\\s*\\}|\"\nPATTERN[2]=\"|\\s*while\\s*\\([^)]+\\)\\s*\\{\\s*iteratee\\(index\\);\\s*\\}|\"\nPATTERN[3]=\"|\\s*else\\s*\\{\\s*assocSet\\(data\\b[\\s\\S]+?\\}|\"\nPATTERN[4]=\"|\\bcase\\s+(?:dataView|set|map|weakMap)CtorString:.+|g\"\nPATTERN[5]=\"|\\bindex,\\s*iterable\\)\\s*===\\s*false\\)[^}]+?(break;)|\"\nPATTERN[6]=\"|\\s*if\\s*\\(\\!lodashFunc\\)\\s*\\{\\s*return;\\s*\\}|\"\nPATTERN[7]=\"|\\s*define\\([\\s\\S]+?\\);|\"\nPATTERN[8]=\"|\\s*root\\._\\s*=\\s*_;|\"\n\nif [ $ISTANBUL == true ]; then\n  set -e\n  for PTRN in ${PATTERN[@]}; do\n    node .\/test\/remove.js \"$PTRN\" .\/lodash.js\n  done\nfi\n",
    "git clone --depth=10 --branch=master git:\/\/github.com\/lodash\/lodash-cli .\/node_modules\/lodash-cli && mkdir $_\/node_modules && cd $_ && ln -s ..\/..\/..\/ .\/lodash && cd ..\/ && npm i && cd ..\/..\/",
    "node .\/node_modules\/lodash-cli\/bin\/lodash -o .\/dist\/lodash.js"
  ],
  "script": [
    "[ $ISTANBUL == false ]   || istanbul cover -x \"**\/vendor\/**\" --report lcovonly .\/test\/test.js -- .\/lodash.js",
    "[ $ISTANBUL == false ]   || [ $TRAVIS_SECURE_ENV_VARS == false ] || (cat .\/coverage\/lcov.info | coveralls) || true",
    "[ $ISTANBUL == false ]   || [ $TRAVIS_SECURE_ENV_VARS == false ] || (cat .\/coverage\/coverage.json | codecov) || true",
    "[ $SAUCE_LABS == true ]  || [ $ISTANBUL == true ] || cd .\/test",
    "[ $SAUCE_LABS == true ]  || [ $ISTANBUL == true ] || $BIN $OPTION .\/test.js ..\/lodash.js",
    "[ $SAUCE_LABS == true ]  || [ $ISTANBUL == true ] || [ $TRAVIS_SECURE_ENV_VARS == false ] || $BIN $OPTION .\/test.js ..\/dist\/lodash.min.js",
    "[ $SAUCE_LABS == false ] || rm -rf .\/node_modules\/lodash",
    "[ $SAUCE_LABS == false ] || ($BIN .\/node_modules\/lodash-cli\/bin\/lodash modularize exports=node -o .\/node_modules\/lodash && node .\/node_modules\/lodash-cli\/bin\/lodash -d -o .\/node_modules\/lodash\/lodash.js)",
    "[ $SAUCE_LABS == false ] || $BIN .\/node_modules\/lodash-cli\/bin\/lodash core -o .\/dist\/lodash.core.js",
    "[ $SAUCE_LABS == false ] || npm run build",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"lodash tests\"     runner=\"test\/index.html?build=..\/dist\/lodash.js&noglobals=true\"     tags=\"development\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"lodash tests\"     runner=\"test\/index.html?build=..\/dist\/lodash.min.js&noglobals=true\" tags=\"production\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"lodash-fp tests\"  runner=\"test\/fp.html?noglobals=true\"                                tags=\"development\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"underscore tests\" runner=\"test\/underscore.html?build=..\/dist\/lodash.js\"               tags=\"development,underscore\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"underscore tests\" runner=\"test\/underscore.html?build=..\/dist\/lodash.min.js\"           tags=\"production,underscore\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.js\"                 tags=\"development,backbone\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.min.js\"             tags=\"production,backbone\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.core.js\"            tags=\"development,backbone\"",
    "[ $SAUCE_LABS == false ] || $BIN .\/test\/saucelabs.js name=\"backbone tests\"   runner=\"test\/backbone.html?build=..\/dist\/lodash.core.min.js\"        tags=\"production,backbone\""
  ],
  ".result": "configured",
  "global_env": [
    "BIN=\"node\" ISTANBUL=false OPTION=\"\"",
    "SAUCE_LABS=false SAUCE_USERNAME=\"lodash\"",
    {
      "secure": "tg1JFsIFnxzLaTboFPOnm+aJCuMm5+JdhLlESlqg9x3fwro++7KCnwHKLNovhchaPe4otC43ZMB\/nfWhDnDm11dKbm\/V6HlTkED+dadTsaLxVDg6J+7yK41QhokBPJOxLV78iDaNaAQVYEirAgZ0yn8kFubxmNKV+bpCGQNc9yU="
    }
  ],
  "group": "stable",
  "dist": "precise"
}

Nodezoo:

{
  "name": "nodezoo",
  "url": "https:\/\/travis-ci.org\/rjrodger\/nodezoo",
  "id": 376765,
  "slug": "rjrodger\/nodezoo",
  "active": true,
  "description": "A microservices workshop for the Seneca framework.",
  "last_build_id": 110126578,
  "last_build_number": "37",
  "last_build_state": "failed",
  "last_build_duration": 20,
  "last_build_language": null,
  "last_build_started_at": "2016-02-18T13:56:04Z",
  "last_build_finished_at": "2016-02-18T13:56:24Z",
  "github_language": "JavaScript",
  ".result": "not_found",
  "language": "ruby",
  "group": "stable",
  "dist": "precise"
}

Proposed object:

{
"name":
"url":
"id":
"group":
"active":
"buildState":
"lastBuilt":
}
@mcdonnelldean
Copy link
Contributor

@matt-oc Can you run them through a pretty printer and use the code formatting, otherwise I can't read them

@matt-oc
Copy link
Member Author

matt-oc commented Apr 4, 2016

I like ID, Group, Last State and Time, they are valuable and seem to be consistent and likely to have correct information (the language for instance is usually incorrect on Travis)

@matt-oc
Copy link
Member Author

matt-oc commented Apr 4, 2016

@mcdonnelldean I'm trying to find one to give a nice output, bear with me!

@mcdonnelldean
Copy link
Contributor

Also, A link to the latest build

@matt-oc
Copy link
Member Author

matt-oc commented Apr 4, 2016

Yes thats a good point so we have ID, Group, Last State and time, and a URL link to the Travis page.

That output looks a bit better now too 👍

@mcdonnelldean
Copy link
Contributor

Can you make a fake payload at the bottom of your original message, with the ones you think are right, I can edit accordingly then.

@mcdonnelldean
Copy link
Contributor

As a JSON object

@matt-oc
Copy link
Member Author

matt-oc commented Apr 7, 2016

Initial proposed object now implemented

@mcdonnelldean
Copy link
Contributor

@matt-oc Lets leave this and the others open for the time being, just in case we want more from them

@matt-oc
Copy link
Member Author

matt-oc commented Apr 7, 2016

@mcdonnelldean Yep will do, I'm updating the original comment whenever the data is changed so people know where we are with regards to data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants