Talks & Speakers (Google Test Automation Conference)

Talks & Speakers

Contents

1 Wednesday, October the 21st
2 Thursday October the 22nd

Wednesday, October the 21st

10:00 Opening Talk - Pr. Niklaus Wirth

Abstract
The activity of testing is as old as programming. We recall the early days of programming, the techniques available at the time, and the introduction of tools for testing and - mainly - debugging. Then we try to compare the old techniques with the modern state of the art, and to critically assess the progress achieved.

Bio
Niklaus Wirth was born in February 1934 in Winterthur, Switzerland. He studied electrical engineering at ETH (Federal Institute of Technology) in Zürich, graduated in 1959, received an M.Sc. degree from Laval University in Quebec, and a Ph.D. from the University of California at Berkeley in 1963.Wirth has been an Assistant Professor of Computer Science at Stanford University (1963-67) and, after his return to Switzerland, a Professor of Informatics at ETH from 1968 – 1999. His principal areas of contribution were programming languages and methodology, software engineering, and design of personal workstations. He has designed the programming languages Algol W (1965), Pascal (1970), Modula-2 (1979), and Oberon (1988), was involved in the methodologies of Structured Programming and Stepwise Refinement, and designed and built the workstations Lilith, with high-resolution display, mouse, and high-level language compiler in 1980, and Ceres in 1986.He has published several text books for courses on programming, algorithms and data structures, and logical design of digital circuits. He has received many prizes and honorary doctorates, including the Turing Award (1984), the IEEE Computer Pioneer (1988), the Award for outstanding contributions to Computer Science Education (acm 1987), and the IBM Europe Science and Technology Award in 1989. www.inf.ethz.ch/personal/wirth

11:30 Precondition Satisfaction by Smart Object Selection in Random Testing - Yi Wei and Serge Gebhardt, ETH Zurich

Abstract

A random testing strategy for object-oriented software basically constructs test cases by performing the following two tasks: 1) randomly select a method under test (MUT); 2) randomly select or construct objects to feed to the chosen method as either target or arguments. Usually, all the objects that are created for or returned by a MUT are stored in an object pool so they can be reused for future test cases. When working with OO software equipped with contracts, it becomes difficult for a random testing strategy to select objects that satisfy the precondition of the MUT. As a result some methods are never tested because all generated test cases fail to satisfy their preconditions. An evaluation of the object pool showed that the traditional strategy often misses object combinations that do satisfy the MUT’s preconditions. Therefore we keep track of these object combinations during the testing process, and directly select them for MUTs. We call this the smart object selection strategy. We implemented the idea in our testing tool AutoTest for Eiffel. We introduced a predicate pool to keep track of object combinations satisfying preconditions of a certain method. All preconditions appearing in the classes under test are collected into a predicate pool. After each test case run, these predicates are evaluated against the objects that are used in that test case. Object combinations satisfying a given predicate are marked in the pool and associated with that predicate. Later, when a method is to be tested, objects satisfying that method’s precondition predicates (as shown by the predicate pool) can be directly selected. This is called smart object selection. We ran this algorithm for one hour on classes with strong preconditions. Results show that the algorithm is indeed able to test methods whose preconditions were rarely satisfied by the original random testing strategy (which therefore often remained untested). In terms of number of faults, the algorithm finds slightly more than original random testing in general. In terms of kinds of faults, this algorithm finds up to 30% of faults that cannot be found by original random testing in some classes.

Bios

Yi Wei is a PhD candidate in Chair of Software Engineering, ETH Zurich since 2007. He is working on software testing. Before he came to ETH, he received a master’s degree in Software Engineering in Wuhan University, China in 2006. He was an intern in Eiffel Software, California, USA from 2005 to 2006.

Serge Gebhardt is a MSc student at the Chair of Software Engineering at ETH Zurich, where he currently writes his MSc thesis in the area of automated software testing. He previously earned a BSc in Computer Science from ETH Zurich and has been a visiting research student at EPFL (Lausanne, CH).

12:30 Fighting Layout Bugs - Michael Tamm, optivo GmbH

Abstract
So you have unit tests, integration tests, and maybe even frontend tests, and you know that your web application does what it is supposed to do. But what about layout bugs? How do you ensure, that every page looks like the designers wanted it to look like? I will present several proven and some novel techniques to help you automatically check the work of your HTML and CSS programmers. I will show examples of typical layout bugs and explain which techniques can be applied to automatically detect/prevent those types of bugs in the future without human intervention.

Bio
Michael Tamm currently works as System Architect at optivo. He finished his studies of computer science in 1999 and is developing enterprise web applications since then. Through the years he has specialized on quality assurance via automatic testing. Besides being a regular speaker at different computer conferences and author of several computer journal articles, he is also an active committer on the WebDriver project.

14:30 Even better than the real thing - Lessons learned from testing GWT applications - Nicolas Wettstein, Google

Abstract
Web applications are hard to write and even harder to test. GWT - the Google Web Toolkit - simplifies these tasks to some extent and there are still pitfalls that make GWT applications hard to test. This talk will discuss the most important pitfalls and demonstrate how they can be avoided.

Bio
Nicolas joined Google in May 2007. He has contributed to various projects as a specialist for developer testing. His focus includes developing testing tools, conducting performance studies, and advising software developers on writing testable code. He is currently based in Sydney, Australia and works on Google Wave. Nicolas holds a Dipl. Informatik-Ing. degree from ETH in Zurich and joined Google with several years of testing experience acquired in Zurich at Esmertec AG and Greenliff AG.

15:30 Automatic workarounds for web applications - Alessandra Gorla and Mauro Pezzè, University of Lugano

Abstract
We introduce an approach to automatically avoid failures that may derive from unexpected interactions between web applications and APIs. Faults in popular APIs, like Google Maps, may affect many users, who must rely on API developers who fix these faults. Asynchronous communication mechanisms and different priorities between API developers and application users may delay fault fixing and reduce dependability of web applications. Fault repositories and forums often suggests temporary workarounds to avoid or mitigate the effects of open faults. We show that many faults can be indeed avoided with workarounds that exploit intrinsic redundancy of web applications, and we indicate how some important classes of workarounds can be automatically generated, thus reducing the impact of interaction faults between Web applications and API. Successful exploitation of automatic workarounds can lead to libraries of rules reusable across applications.

Bios

Alessandra Gorla is a Ph.D. candidate in Informatics at the University of Lugano (Switzerland). She holds a Bachelor and an MSc degree in Computer Science both from the University of Milano Bicocca (Italy). Her research interests include software testing and analysis, and self-healing systems. Alessandra Gorla works on contextual data flow analysis and on automatic generation of valid workarounds.

Mauro Pezzè is a Professor of Software Engineering and Dean of the Faculty of Informatics of the University of Lugano. He holds a position of professor of software engineering also at the University of Milano Bicocca. Mauro Pezzè is associate editor of ACM Transactions on Software Engineering and Methodology and member of the Steering Committees of the International Conference of Software Engineering (ICSE) and of the ACM International Conference on Software Testing and Analysis (ISSTA). He is a co-author of many papers on software engineering and software testing, and is the co-author of a book on Software Testing and Analysis published by John Wiley in 2008.

17:00 Achieving Web Test Automation with a Mixed-Skills Team - Mark Micallef, BBC Future Media and Technology

Abstract
Test automation is commonly perceived as a technical job, which should be done by technical people. In reality however, a considerable amount of non-technical activities contribute to the creation and maintenance of an automated test suite. Whilst test engineers are arguably competent at non-technical testing tasks, they may not be as competent or even as motivated as people who come from a manual testing background. Furthermore, in the context of most systems, especially web-based systems, automated testing would also benefit from people with primary skills outside of testing. These include developers, information architects, business analysts, customers and so on.This talk explores ways of harnessing the experience and skills of people with different backgrounds and channeling them towards the effective test automation of web-based systems.”

Bio
Mark Micallef has worked in software quality assurance and test automation for 8 years. Having spent time working in the security and finance sectors, he moved to London last year to lead a team of testers responsible for the BBC's News, Sport and Weather websites.

Thursday October the 22nd

09:00 Opening Talk - Dr Alberto Di Meglio

Bio
Dr. Alberto Di Meglio graduated in Aerospace Engineering at the Politecnico di Milano in 1993 and received a Ph.D. in Electronic and Electrical Engineering from the University of Birmingham in 2000. Alberto has worked as Research Associate at the Electrical & Electronic Engineering School of the University of Birmingham from 1995 to 1997 as part of the EC-funded Vertlink communications project. In 1998 he joined CERN where he worked for three years as systems engineer in the Information Technology Division. During that period, Alberto was responsible for a number of projects in the field of networked and web systems. In 2001 he left CERN to fund a software company developing a multi-platform system for the management and monitoring of distributed systems using the WBEM standards. During this period Alberto was Chief Technology Officer and R&D Manager responsible for development, integration and testing of the company software. In 2003, he was appointed by CERN as Software Integration Manager in the Middleware Reengineering Activity of the first EGEE project. At the end of the EGEE project, thanks to the successful results obtained with the integration and testing tools and procedures developed for gLite, Alberto set up the ETICS project, an international infrastructure co-funded by the EC FP7 program for building and testing software on the grid, of which he is currently Project Director. He is a member of the Italian Board of Engineers, a Chartered Engineer of the British Engineering Council, a member of IET and IEEE and a certified ITIL professional.

10:00 Testing Applications on Mobile Devices - Doron Reuveni, CEO & Co-Founder of uTest

Abstract
In the highly competitive world of mobile applications, getting high-quality apps to market quickly can make or break the success of a product. With new applications for Android, iPhone and Blackberry battling for media attention and consumer dollars, the pressure to get apps built, tested and launched has never been greater. But the testing methods that have worked for web and desktop app companies (in-house QA, offshore testing firms, emulators/simulators, beta testers, etc.) don’t meet the QA needs of mobile apps. The testing matrices in the mobile app world have become far too complex. Companies must test their apps across handset makers, handset models, wireless carriers, OS and browsers. And beyond that, they MUST account for location-based testing – a task that has proven difficult and prohibitively expensive for even the most mature and sophisticated teams. This unmet challenge calls for a new approach. Crowdsourcing is uniquely suited to meet this challenge. By tapping into the diverse crowd of testers who operate outside the lab environment, companies can build a virtual testing team who meets their coverage requirements. This session will examine the growing trend of crowd sourced testing for mobile applications, as well as addressing the benefits & challenges of this model, compared with conventional testing options. In this session, Doron Reuveni will address:Alternative Testing Methods and Why They Fall Short of Meeting Mobile’s “In-The-Wild” Testing Demands:In-House Testing Outsourcing/Offshoring Emulators/Simulators Beta Testers The Crowd sourcing Option: Building an On-Demand Testing Team The Complimentary Role a Community Plays in Mobile Testing Crowd sourcing Challenges: Maximizing Quality, Achieving Coverage, Crowd Control Real-World Case of a Company Utilizing “Crowds” to Test & Launch Mobile Apps

Bio
Doron Reuveni co-founded uTest in February, 2007. Since its inception, uTest has provided functional and usability testing services through a community of QA professionals that includes 20,000+ testers from 157 countries around the globe. He is a published author and expert in testing methodologies for web, desktop and mobile applications. Prior to uTest, Doron was the Senior Vice President of Technology at Enigma, Inc., delivering large-scale software implementations through his global team of product managers, developers, QA professionals and project managers. He has more than 20 years of experience delivering software applications to Fortune 500 companies as well as young, agile startups.

11:30 JsTestDriver - Jeremie Lenfant -Engelmann, Google

Abstract
The proliferation of JavaScript unit-testing frameworks in the JavaScript community shows that no one has yet found the magical combination of features to make JavaScript testing a no-brainer. We like to believe that JsTestDriver will change that by redefining JavaScript testing frameworks and as a result will deliver something which at first glance seems impossible. As a developer of JavaScript I want to write my production and test code in my favorite IDE. When I make a code change, I want a save action to trigger a rerun off all of my tests on all browsers and platforms and report the results under 1 second. All of this without ever leaving my favorite IDE. JsTestDriver achieves all of the above by capturing any number of browser slaves from multiple platforms. In order to run at high speed we do not render our results in the browser but report the errors on the command line. JsTestDriver also loads and parses JavaScript files eagerly, only reloading files that have changed. JsTestDriver can execute hundreds of tests per browser and execute them on all of the captured browsers in parallel. Thanks to these practices we achieve extremely high throughput. We want to be a test-runner platform and allow others to build assertion frameworks on top of JsTestDriver. This way the open-source community can focus on building a better way of testing and not worry about how to run the tests across all of the browsers, platforms and in a continuous build environment. http://code.google.com/p/js-test-driver

Bio
Jeremie Lenfant-Engelmann works for Google and is the lead developer of JsTestDriver.

12:30 Selenium: to 2.0 and Beyond! - Simon Stewart, Google & Jason Huggins, Sauce Labs

Abstract
As astute attendees of previous GTACs have observed, there's a lot in common between Selenium and WebDriver. They're both fantastic tools for automated testing of web apps, but they do their thing in radically different ways. As keen followers of the projects, you may have heard of the plan to merge these projects. What does this mean? And, perhaps more importantly, why should you care? We'll explain how Selenium and WebDriver work, then look to the future and explain what Selenium 2.0 will be. We'll tell you what we plan to do, how we plan to do it. We'll do this by taking a Selenium1.0 script and migrate it into the future of 2.0. Wish us luck!

Bios

Simon Stewart lives in London and works as a Software Engineer in Test at Google. Simon has previously worked at ThoughtWorks, specialising in Agile and Test Driven Development. His Open Source contributions now largely centre on WebDriver, the browser automation project now in the process of merging into Selenium, which he is the technical lead for. It has been said before that Simon enjoys writing better software and beer, sometimes at the same time. This is still true.

Jason Huggins cofounded and leads product direction at Sauce Labs, which offers cloud-based commercial support for Selenium. Before SauceLabs, as a Test Engineer at Google, Jason supported the grid-scale "Selenium Farm" for testing Google applications, such as Gmail and Google Docs. Before Google, Jason was a software developer at ThoughtWorks in Chicago. There he created the Selenium testing framework out of the desperation to cross-browser test a new in-house time and expense system. Jason works and lives in Chicago, Illinois, USA. Besides enjoying programming with Python, JavaScript he likes to hack on Arduino-based electronics projects.

14:30 Score One for Quality! - Joshua Williams and Ross Smith, Microsoft

Abstract
Research into the generation gap between current managers from the Baby Boomer era and the incoming group of Gen X, Gen Y, and Millennials shows that the use of video games are a significant distinction to separate the two. Those generational differences indicate the need, as we seek to understand this research, for experimentation in how test teams find bugs. We found games to be a very powerful mechanism for communicating organizational priorities effectively and quickly. People engage quickly, and games can drive QA behaviors that help improve both product quality and employee morale. This creates a virtuous cycle where standard productivity metrics improve as engagement improves. The game discussed here introduces a new level of quality into the localization efforts by using “the crowd” – a diverse worldwide employee base in this case. Based on our experience, we believe that “Games at Work” or “Productivity Games” carry a huge potential for influencing not just the software engineering workplace, but all industries and employee populations.

Bios

Joshua Williams is a senior software quality engineer at Microsoft, currently working on the WindowsCore Security Test Team. Over the past 14 years, he has contributed to the development, testing and shipping of Windows Operating Systems. Across at least nine different releases, Joshua has been an individual contributor, lead and manager across a variety of component teams and technologies. These include the Windows International Team, focusing on versions of Windows besides English and the US Band 1394 teams, shipping support for numerous external device classes, including the first USB 2.0 (Hi-Speed) devices. More recently, Joshua has focused on large-scale test automation frameworks and development lifecycle tools to improve test and development productivity. As part of the Windows Security Test Team, Joshua has researched new ways to engage team members, working to improve the quality of work, but also the quality of life of the team. Joshua loves all kinds of games, and brings that passion to his work on “Productivity Games”.

Ross Smith is a Director of Test at Microsoft and has been in the software industry for over 20 years, developing and testing software on everything from mainframe systems to handheld devices and PC’s. He began his Microsoft career in Product Support in 1991 and has been an Software Developer in Test, a Test Lead, Test Manager and Test Architect. He has been a long-time member of Microsoft’s Test Architect’s Group, and has worked on many versions of Windows and Office. He is one of the authors of “The Practical Guide to Defect Prevention” and holds five software patent sfor quality-related ideas. Over the last couple years, he has nurtured a management innovation initiative called 42Projects (www.42projects.org), aspiring to create cultural change by using trust and games to improve the way teams work. Ross was a guest poster on Google’s Test blog, and you can find his writing on Productivity Games at http://productivitygames.blogspot.com

15:30 Automated Performance Test Data Collection and Reporting - David Burns and David Henderson, smartFOCUS DIGITAL

Abstract
Web applications are growing in size and complexity with every new release. The addition of slightly more JavaScript and HTML can lead to the site becoming sluggish without an obvious cause. Fortunately more organizations are taking note of the correlation between site speed and profitability. Without suitable tools, developers are left stabbing in the dark to resolve performance issues until the application feels faster. Luckily there are many tools available, such as YSlow, that can help you through the process of measuring the performance of your application. However gathering this data manually can be time consuming, laborious and prone to human inconsistencies. To illustrate the need for better performance information, we will outline the state of performance testing within the development cycle as carried out by many development teams around the world. We will then discuss the requirements for the system that has been created and implemented as a result of our research and development. The data gathering infrastructure, the tools used to create it and the scope of the data that is collected will be shown with solutions to problems encountered along the way. Performance statistics of a page, kept in a database, provide little useful information in isolation. When put into context with data from other pages and previous builds, the performance statistics suddenly become invaluable. To display this data in a digestible and comparable setting, a reporting portal will be demonstrated and its place within the development lifecycle explained. The final component of the system is the integration with the Tester's Heads-Up Display (T.H.U.D.). This is a plugin that aids rapid diagnosis and reporting of bugs by overlaying performance data on a specific page, as well as providing access to source control and bug tracking systems. This presentation will show how 'Automating Performance Test Data Collection and Reporting' has improved the awareness of web performance issues within our company. It has provided the evidence required to instigate changes and measure their impact. An average 75% reduction in primed page size has been achieved as a direct result of the system's introduction.

Bios

David Burns and David Henderson are both members of the development team at smartFOCUS DIGITAL, working on their SaaS solution.

David Burns is the Lead Test Engineer, working on the web accessible parts of the system. He is the test automation champion for smartFOCUS and heads up the Test Design Authority within the group trying to find best practices in testing of smartFOCUS applications. David is an active blogger at http://www.theautomatedtester.co.uk.

David Henderson graduated from the University of Southampton with a first class Masters in Engineering in 2007. He is a developer working on the front end development mainly dealing with JavaScript and C#. David is currently tinkering with the Android platform in his spare time, looking to write the next killer app.

Both have an unhealthy obsession with measuring and improving the speed and weight of the user experience.

Subpages (1): Slides