My e2e Top 3

Having an automated end to end test suite can be a super comforting assurance that your application is in a ready state.  Nothing says 'ship it' like having just run a user-like test against every independent feature in the application with no mocks.  Maintaining that test suite can become like battling a hydra if the tests are not properly built though.  I was on a team a few years back that spent about 40-60% of the day maintaining existing cucumber tests.  That is counterproductive and just plain painful.  People get cranky about that sort of thing.

Before I dive in let's lay out a working definition of End to End tests for this post.  I define End to End (e2e) tests to be those that interact with the live application through the user interface and very rarely, if ever, mock functionality regardless of who maintains that functionality.  Tools such as Cucumber, Protractor, Geb, and RSpec are commonly in used to write these tests.  Techniques such as ATDD or BDD generally yield e2e tests.  E2E tests are at the top of the testing pyramid and there are far fewer of them than unit tests.

OK, academia complete...on to the good stuff...

Thing One - specify behavior, not implementation

You would think that a practice rooted in the name Behavior Driven Development would make this point obvious, but we continue to get this wrong.  Any time you find yourself using words like "Page", "Button", "Link", "Text Box", "Drop Down", "Radio Button", etc. in an e2e test specification it is a smell and you should consider whether or not there is a better way.  The reason may not be obvious at first, but what occurs with these tests is that they become increasingly brittle over time.  As the application matures and the implementation morphs these tests become a maintenance headache.  Although there is more to it than just names, they honestly best illustrate the difference.

Tests that should serve you well over time

• The XYZ Application should allow the user to enter and store address information.
• The User Information feature should allow the user to navigate to the Item Ordering feature.
• The user should be able to view all previously ordered items in History.

Tests that will eventually make you wish you never wrote them

• The XYZ Application should have an address form with Street Address, City, State, Postal Code fields.
• The user should be taken to the Order Page upon clicking the Place Order Button on the User Information page.
• The user should see a table including Order Date, Item Name, and Order Cost on the History Screen.

Thing Two - use abstractions to isolate implementation awareness

In most cases these days this really boils down to using the page object pattern.  Regardless, the important point is that there should be exactly one place in the entirety of your e2e test suite where location of a given UI element takes place.  There should be an object (used very loosely here) whose single responsibility is knowing how to get a hold of the UI and interact with the elements.  Any e2e specification should use these objects to locate and retrieve UI elements and and never do so directly in the spec.

For example, there is a fantastic explanation of how to accomplish this in AngularJS/Protractor over on the ThoughtWorks blog.  I implemented this pattern over the last few months with a web development team at a client site and we have all been very pleased with the outcome.

Thing Three - don't test against specific data, but if you must, control it

I cracked open a failing e2e spec the other day that was expecting 4 rows of data in an HTML table  and only found 3.  I didn't see anything in the 'arrange' portion of the spec that was loading the data so I scrolled up to the 'beforeEach'.  Nothing there either.  Hmmm...OK...Pardon me while I flip a table!

In all seriousness, if for some reason you are compelled to look for specific data in an e2e spec, be sure the spec is managing the test data.  A test that is written to be dependent on data from a completely separate process is going to fail and probably sooner than you think.  Better still is to try and keep most of the e2e specs data independent.  If the behavioral assertion can be vague enough to simply check that there is data, for instance, then let it be less precise for the sake of always being accurate.  There should be a unit test somewhere that handles making sure inputs equal outputs and the name doesn't end up in the address attribute of an object.

A data independent approach can occasionally leave holes in assurance.  For instance, if the wrong data field is bound to the view and the e2e test doesn't inspect the value of the data in the view the mistake can go unnoticed.  In these specific cases decide what is least painful for your team, but try to keep control of things so that the test doesn't fail next week.  Maybe a mock is in order, or an inserted row in the back end database, or perhaps a regex to ensure the data at least meets certain criteria.

Thanks for reading this far.  Happy testing!

Why Are We Here?

So, I'm getting old...or at least older.  I made it to 40 this year and lately find myself asking all sorts of decidedly silly questions that 25 year old Don did not care about in the least.  Recently it was how does a software developer really express value? Deep, right...  I am sure a question of this sort immediately preceded the invention of Scotch, in fact.

The obvious answer is producing code.  The path from code to business value is an easy one to map out.  I do see people wondering off that path regularly, but we know it's there and can usually find our way back.  That's likely a topic for other posts.

A quick second in my responses...to myself...was 'making more developers' which I further refined to 'making other human beings better software developers'.  IMHO, this is how we advance as a profession.  And that, I think, is at the core of the matter.  I would like to know that in the grand scheme my actions in this decade have contributed to a positive outcome for the generations of developers that follow.

On a soon to be related note, a few years ago I went to work for a great company as a consultant.  The owner of my company has since beaten into my head the idea that if we do not raise our game beyond solid engineering we will never be able meet the growing demand for software.  To his way of thinking, unleashing the creative and innovative mind of the human beings that write code is the only way forward.  And I tend to agree.

At the intersection of these two ideas; developing human beings into happier, more productive software craftsman and providing an environment for creativity, is an interesting place.  It leads to several changes in behavior.  All of the sudden you have to get to know people on a different level before you hire them.  You have to provide a physical environment that evokes more than efficiency.  There's a whole lot more touchy, feely humanity involved.  And that scares people because many of the usual categories and containers and even tactics we use with humans in business don't apply any more.

A few days ago I warned my management team to be ready for things on the client site to get weird.  We are making some fairly massive changes there and I didn't want them to be surprised when the calls came from people who were confused.  The same advice applies to all of us who find and serve software developers.

As coaches and leaders, be ready for things to get weird.  Actually, make sure they get a little weird.  Know that people can code, but also get know what makes them tick.  Maybe you really do need better furniture, or team movie over lunch, or legos at your pairing station.  Maybe an outlet like 'lightning talks' once a week covering important things like the superiority of music recorded on vinyl is precisely the thing that will send developers back to their IDEs ready to create code like a boss.

My hope is that if we start looking at the people we work with as a creative force and add that to their career offering we can advance the profession.  If it is expected that we grow in our creative outlets as well as our technical tool chest we should unleash some pretty non-standard solutions to the really hard problems coming at us as business becomes more and more dependent on software.  Put more simply I guess I wouldn't expect 'out of the box' answers from people I never let out of the box.