July 11, 2019 // By Josh Wencl
Managing test data is almost always a necessity for any automation project. For this I’m just going to define test data as:
Any type of data required by your test to run.
This could be any of the following:
- Typical faked data for a test run (People names, fake email addresses)
- Data driving documents (CSV, Datatables)
- Environment configurations (URL, client ids, client secrets, browser versions)
- Integrated environment (Usernames, passwords, real email addresses)
A common thing for many testers to do is to store these types of data alongside their automation code, and for a few of these I find that perfectly acceptable.
Faked data like people names where the only goal is creating some randomized “realistic” looking data, or data driving tests from CSV docs, these are fine. These types of data define behavior, or the “what” or “why” of your test run.
The types of data you should not couple with your tests as the “which”:
- Environment configurations (URL, client ids, client secrets, browser versions)
- Integrated environment (Usernames, passwords, real email addresses)
“Which version of Chrome do I run my tests against?”
“Which environment am I trying to connect to?”
If you tightly couple your test code with your configurations, then you aren’t allowed to make changes to your configurations with changes to your code.
Take the following example. Your test code contains a series of classes where you define your application URL, client secret, and client id, the code might look like this:
Environment QA = new Environment(“www.qaenvironment.testproject.com”, “client_secret”, “client_id”);
Environment DEV = new Environment(“www.devenvironment.testproject.com”, “client_secret”, “client_id”);
You can quickly check what environments you’re using when you’re running your tests locally, but when you compile that code into a .dll or a .jar as part of your release artifact you have effectively frozen your code.
If another environment is added, you would have to update your test code with the added environment. Depending on your branching strategy, this could be more complexity than you need.
Containerization also makes it incredibly easy to setup and tear down new environments, which your tests should be able to take advantage of. In Cloud Foundry you can set up the application route and pass that in as the route your tests use.
If you hard code your environment configurations, then you lock yourself out of this capability.
Integrations can change as well, passwords can expire, user accounts change, and again its not something you want to be tied to.
The most ideal setup for automation is to have your tests be as data agnostic as possible. If they can dynamically create user accounts or setup your own email server (spin up a new email server with docker), you should strive to do it.
Failing that, have a layer of separation. If you really need to keep hard-coded passwords somewhere, pass them in as command line arguments in your test run. AzureDevops and Jenkins can both store those variables.
You can store your information in a retrievable service you pull through a rest call, or in a database you access through SQL.
Just whatever you do, don’t hard-code that into your test code.