February 8, 2019 // By Paul Grizzaffi
You’re the person responsible for your team’s initial automation suite. Using your desktop development computer, you created all the scripts, tested them extensively, and now they are ready to be used. They are based on your existing smoke test suite, so you plan to run them each time there is a deploy; the test team won’t need to perform that activity anymore. The automation is a rousing success, saving multiple hours of tester time every week! You plan a well-earned vacation; you’ll only be gone for a week.
Who’s going to run the automation in your absence?
Your infrastructure for automatic script execution isn’t ready yet, so a lucky team member is conscripted into executing the automation on your behalf. As part of your due diligence, you help your team member get the automation downloaded or installed on their desktop development machine. Now, you have them run it just to, you know, be sure they can run it. They can’t. The scripts fail to execute. You work with your team member to sort out the issues:
- They need a specific directory in a specific location on their local disk; they create the directory.
- You have a file path that’s hardcoded to your user directory; you make a code modification and redeploy the automation.
- You make a call to an external program that’s not one their desktop; you help them get it installed appropriately.
Finally, the scripts run, but they don’t pass; perhaps there is an issue in the application. To assess the situation, you run the scripts on your machine. They pass as expected. After further investigating, you discover that your team member’s machine is an older, less-powerful machine than yours is; it runs the automation slower causing the application to timeout while waiting for responses.
Oh crap! You’re supposed to leave for vacation tomorrow…
While this specific story didn’t happen, I’ve seen parts of it happen and heard of other parts of it happening. The situation here is very much akin to “works on my machine”. The original author developed the automation to run on their machine without consideration of others needing to run the automation; in this case, “others” might be other team members or some automated system such as a continuous delivery pipeline activity. In other words, the automation was not developed for portability.
Portability has two main facets:
- Execution environment; the story above mostly shows this facet. Automation must not be bound to a single computer.
- System under test. Most of us have more than one environment against which we need to test, meaning we usually have more than one environment against which to execute our automation. In most cases, automation must not be bound to a single test environment.
Here are some potential danger signs of which we should beware:
- Anything that’s hardcoded. Hardcoded values limit our ability to be flexible. Most language ecosystems have idioms for handling application configuration; it’s not different when that application is automation. Learn and exploit the idioms for your language’s ecosystem.
- Code that relies on specific file paths to exist. While not necessarily something to unconditionally avoid, we need to understand that the more specific the file paths are, the more brittle the automation will be. A better approach is to use paths that are relative to where the automation software is located. Also, when possible, make the automation create directories instead of assuming they already exist.
- Code the relies on external applications. Again, it’s not
necessarily something to unconditionally avoid, but we need to make provisions for installing that necessary software and give helpful error messages when the application can’t be found.
- Code that relies on individual user configuration. Though this can be a powerful capability, we must realize that provisioning each user with the necessary attributes is not a zero-effort activity, especially when we need to work with other organizations for user provisioning.
It may not be feasible to develop our automation to be executable on every computer in the company, but developing for portability increases the number of team members that can participate in the automation endeavor and portability is the first step into automated execution as part of a continuous delivery pipeline.