March 26, 2019 // By Paul Grizzaffi
I was a speaker at the 2019 Automation Guild Conference. As part of the conference, I participated in a live Q&A session, but we ran out of time before I answered all the questions. I decided to blog the answers to some of those questions.
When we talk about BDD, the Gherkin itself is a layer. So, in that case, what could be an ideal pattern for layers? I guess too many layers will make it tough to maintain.
As readers may know, BDD is a hot-button topic for me. Though I wrote about it at length in this article, in the spirit of TL;DR, my stance is that BDD is a framework to attain shared understanding about our user stories or work items, the automation piece is merely a convenience. That’s not to say, however, that the convenience isn’t valuable, it’s just not the primary value.
For the sake of this specific question, I’ll just address the implementation of “the convenience”, better known as “the automation”.
Much like the question addressed in my previous blog post, the question posed above has two parts. The first part is “what could be an ideal pattern for layers” when using BDD tools. Generally, there is no such thing as “ideal”, only the “most appropriate for my situation”. That said, I’ve used the layers below with two different organizations:
For those not familiar with it, SpecFlow is basically the .Net equivalent to Cucumber. To map the layers above to the layers from my Automation Guild talk:
- SpecFlow Feature Files, Gherkin = Scripts
- SpecFlow Step Files = Middle layer
- Page Objects = Lower Layer
Notice that in the mapping above, the SpecFlow files are the implementation of the script and middle layers; those files don’t reside on top of those layers.
This brings us to the second part of the previously stated question, “too many layers will make it tough to maintain”. Yes and no; we’ll take the “no” part first. As mentioned in the previous paragraph, my implementation replaces other layers’ implementations with SpecFlow, therefore, no additional layers are added, so we should not have “too many” layers. If, however, we decide to implement on top of the rest of our layers, we might have “too many” layers; this is the “yes” part of the previous scenario.
To reduce the impact of “too many” layers, each layer we add to our automation stack must provide some sort of value. If there is value to implementing SpecFlow on top of your existing layers, then, do so! To address the “tough to maintain” part we need to understand that this frequently has to do with the challenge, hence the cost, triaging automation script failures. This part of the maintenance cost comes from the difficulty of diagnosing a failure when we can’t get enough detail on where and why an error occurred. The general cure for this difficulty is adding appropriate logging at each layer then allowing that logging to trickle up to the user so that they have a good idea of what the failure meant at each layer. For example:
Notice that at each layer a message was added, and that message was representative of what the error meant at that layer. No errors or messages were obfuscated or eliminated. Having this level of transparency is essential to reducing the maintenance cost of triaging failures.
I hope this answer was not too vague. If you did find it so, my apologies, but you’re in luck. My talk at the Spring 2019 STPCon is about creating an automation stack. You can find the details here and you can contact me for a discount code! You can also check out a bit of an audio preview at that link. If you’re in the Dallas area, I’m giving a “dress rehearsal” of this talk at the March 28, 2019 Ministry of Testing Dallas Meetup.