Testing Thoughts

• End2end or CUJ (Critical User Journey) Tests:
  • purpose: information signalling/monitoring
  • should not be able to block a merge or deploy programatically by failing
  • should be in its own repo should be in same repo
    • for the time being, copy things if reusing for other service rather than centralize, e.g. Job Manager
    • clear it’s Terra test, not Terra UI test
    • Saturn team should own it, QA should feel ownership as well
    • npm install -D github:DataBiosphere/terra-test#commit-ish
      • npm ln ../terra-test for local dev
    • separately delivered
  • should go and click things on external sites, overriding return URL's when necessary (12/5/19)
• general (for end2end or unit tests)
  • data-test-id not ideal
    • moves further from realistic testing
      • users aren’t using that prop, they’re reading text
    • gets littered around the codebase, rarely cleaned up
  • alternative: use content when possible
    • buttons based on text
    • coherent strategy for identifying icons
      • name available in the dom to use as selector
    • where that’s insufficient, figure out ways to improve app so it is sufficient
      • intersection with accessibility
      • if it’s not findable by a test, it’s not findable by a screen reader
        • aria- features, etc.
        • e.g. list of cards and want to open the menu for a specific one:
          • desired description to test: within card that contains text “x”, click the menu icon (css selectors + jest selectors)
      • concerns:
        • if you change text the test will break
          • it should, this is when the test should be updated
          • not expected to happen often
          • false positives from this are more useful than false negatives from using data- prop abstraction
        • waiting
          • avoid arbitrary timeouts whenever possible, wait for element to appear or disappear
          • when not possible, fix the app so it is.