Luke Closs on Open Web Testing and Test Automation - Roland's Notes

After Luke Closs's presentation on Test Automation and Open Web Testing, I am much more up to date and aware of automation testing issues and how one would strategize it. This talk alone makes the conference worthwhile for me. Check out my notes after the jump.

  • Business driven Development leads to testing what is of value to your system
  • Tests are pretty abstract aka very very simple scripting that somehow have to be translated to somethng that will do the tests you want.
  • "Sophos Style" - use YAML and perl
  • "Wiki test Style - SocialText is a wiki company so use wikis i.e. 1 wiki page per test, written by Dev & QA, tests are in the form of wiki tables
  • when QAing web pages have developers put unique ids in HTML pages
  • No conditionals or looping in test automation
  • Separate Test Logic from Test Code using a tool like cucumber (for Ruby)
  • This helps with risk management
  • There are no "Best Practises" depends on context!
  • 5 Guidelines:
    1. Keep up with Development - writing tests can't be isolated, has to be done in sync in the natural flow of developing the software
    2. Make Tradeoffs with intention - strategize your gestalt of testing - Sunny day - 95% normal, automate this; Cloudy/Rainy day - manual testing for this perhaps? or automate this if cheap or strategic
    3. Do the best with what you have - i.e. use the talents of your team, be resourceful with your team, or embrace developer energy, put people in a position they get excited about
    4. Balanced Breakfast - look at multiplicity of strategies and pick what's best for the context and business
    5. Pay Attention - real costs here! automation isn't cheap - e.g. 4 hours to automate, 1 hour to test, do regular team retrospective, reflect on lessons learned and how to improve i.e. periodical reflection, refactor your test
  • History of Socialtext Testing
    • early days unit tests, manual testing, dogfooding (problem since customers used IE unlike Socialtext team
    • 2nd phase t:selenium ide: easy to record but need to clean up best for learning selenium commands or
    • third: wikitest revolution - aka Keyword Based Testing based on Selenes; express tests as a wiki table more abstraction, less duplication, standardization, requires test developer to write software
    • interns - automate everything over summer 6000 tests; automation still lags development; no more regressions, can't remember the last time they shipped a bug to production
    • great depression - qa layoffs - change our practice ; devs will write sunny day automation, QA will write rainy day automation and do manually. QA focusing on exceptions is a win!