Trying to be Agile

dilbert user stories

In my current company, we are still in the process of learning how to become Agile. It is a phase of confusion on how to overcome obstacles like the one depicted in the above Dilbert comic strip.

So, I have been reading and reading. I also picked up a book on “Agile Testing” by Lisa Crispin and Janet Gregory, and I am going through it with my testing team.

The questions that I need answered are –

1. People are resistant to change. How do we inspire them to adopt Agile behaviour without forcing it down their throats?

2. How do you handle someone who comes late in daily huddle almost every morning? Is that person supposed to be thrown out of the team?

3. How do you make the whole team feel responsible for quality when the programmers have made it a habit to leave all testing for the testers?

4. In our teams here, we have programmers specialized in particular sections of the code. Now, if we have a high priority task in one section (worth one or more sprints), and one lower priority task in another section. Tasks are taken up to give the full high priority section to the particular resource. The other resource is asked to work on the lower priority. What is the correct way? Can we improve?

5. We are struggling with the attitude that scrum is only for projects that do not get critical, urgent requirements frequently. How do we change this?

Maybe, as we finish the book, we will get answers to the above questions. Till then, I would love to hear any thoughts that you would like to share about them.

Why I am signing the petition to stop ISO 29119

There is a movement being started to stop ISO 29119. If you are even a wee bit serious about your profession, you should be knowing about it. Here’s all about it. Read it if you want to know more, or just skip to the end and sign the petitions.

James Christie gave a presentation in CAST 2014 on Standards – promoting quality or restricting competition? He talked about how some organizations are trying to enforce testing standards for their own economic gain, while victimizing the software testing profession. He also stated how the standards promote excessive documentation over actual testing, thus affecting the quality of testing by taking away time from the process. At a time, when we are trying to implement Agile, and moving towards exploratory testing, heavy documentation and rigid processes make no sense. And, to force it down the throats of everyone in this profession should not be tolerated.

As Fiona Charles puts it “People on Agile projects will struggle with the conflicting demands of their projects and the standards.”

There is a reason why I have not taken the certification. Why should I pay so much money for something that won’t add any value to my work, except a bunch of words I might never need to use? I know this for a fact that being certified does not say anything about the tester’s expertise or potential. Most current certifications, for a fact, can be easily passed by rote learning. But, would that help? Rote learning never helps. How much do you remember of the history lessons from your high school? You did pass the exam.

I have known many testers who are certified, but do not have a clue about testing, and are low performers. I have also known many testers, who are awesome at their work, have high potential, are high performers, and are not certified. That is why I ignore the certification status of people I interview, and instead focus on finding out myself about how much they are interested in testing, and how well they know their craft.

This is what Karen Johnson says about wrong hiring based on blind belief in certifications:
“For several years when I was working as a test manager, I interviewed and hired numerous people. Once during an interview, a candidate pulled out a certificate and held it up for me, telling me he should be hired instantly due to his certification. I asked that he put the certificate away and talk with me. After discussion, it was my assessment that the candidate had little working understanding of how to test a product. Perhaps he had memorized material or had someone else take the exam for him – whatever had been the case; there was no evidence that the candidate would be able to perform well. As this event took place some years ago, I will refrain from trying to recall more specifics.
Given this same scenario, if I did not have a testing background but needed to hire someone, perhaps I would have been convinced based on the certificate that the candidate was equipped for the job.

If a company is hiring testers and the person or persons interviewing do not understand testing (which is often the case with HR), I do not believe that hiring person is qualified to make a decision – or even qualified to establish a pool of candidates for others to interview. This is where certification is dangerous – at first glance, it sounds good, it seems like it should provide some assurance that a person is qualified. But as I have found, having a certificate does not provide evidence of a person’s knowledge or skill.”

If you read my blog, you will know I am not against learning. In fact, I am very passionate about learning. I keep reading books and blogs to learn more about my profession. But, I want to learn out of curiosity and interest, not out of fear. That is what this standard tries to instill – the fear of losing your job or validity as a software tester if you have not paid the bucks to get certified by rote.

If the standard is accepted, companies will start believing that testers without a certification are not good, which is far from the truth. They will start over-depending on tester certifications to judge the quality of a tester.

Iain McCowatt says “Standards in manufacturing make sense: the variability between two different widgets of the same type should be minimal, so acting in the same way each time a widget is produced is desirable. This does not apply to services, where demand is highly variable, or indeed in software, where every instance of demand is unique.

Attempting to act in a standardized manner in the face of variable demand is an act of insanity: it’s akin to being asked to solve a number of different problems yet merrily reciting the same answer over and over. Sometimes you’ll be right, sometimes wrong, sometimes you’ll score a partial hit. In this way, applying the processes and techniques of ISO 29119 will result in effort being expended on activities that do nothing to aid the cause of testing.”

Karen also explains this in detail in her blog where she says that the knowledge required by an e-commerce software tester, a BI software tester, and a Medical device tester are vastly different, and can not be covered by a generic testing certification. The testers would instead do better to learn more about their domain, and to get domain-specific knowledge.

ISO 29119 claims that it is “an internationally-agreed set of standards for software testing”. But, it is not “internationally-agreed”. In fact, many influential software testers, and the experts that I look up to, are against it.

Karen Johnson – My Thoughts on Testing Certifications
Fiona Charles – Why I oppose adoption of ISO 29119
Iain McCowatt – Stop 29119
Keith Klain – The Petition to Stop ISO 29119
Michael Bolton – Rising Against the Rent-Seekers
James Bach – How not to standardize software testing

If you agree with the above view point, and are interested in saving our profession from vandalism, please sign the below shared documents:

The Professional Tester’s Manifesto

The Petition to ISO

Other articles of note –
ISO 29119 Debate
On ISO 29119 content

Issue in auto suggestions when multiple words of the same record start similarly

similar multiple words in a record

This one adds to my list of test cases for auto suggestion functionality. It really depends on the implementation whether this test applies or not, but it is still good to add to your check list.

In our case, auto suggestions are displayed using keywords mapped to the record. There is one keyword for each word in a name.

Example: “New Delhi” should show up as a suggestion when either “New” or “Delhi” is typed in.
So, there are two records in database – one with “New” as a keyword, and one with “Delhi” as a keyword.

But, what happens when words in a single name start similarly.

For example, there is a city named “Tarn Taran”. Both words in the name start with “Tar”.
When I type in “Tar”, both keywords “Tarn” and “Taran” will get matched, and if it’s not handled in the code, Auto-suggest list will display two records for “Tarn Taran”.

How use of Regular Expressions affects Website Performance

A few months back, I inadvertently came across a bug related to regular expressions. It was a new learning for me that improper regular expressions can cause catastrophic performance problems.

Let me first give you more details about the bug I found.

The form I was testing had a text box “Special Note” with maximum character limit set as 500.
When the form was submitted by the user, a regular expression was used to check that the text did not contain any Email address.
Everything worked fine when I entered strings with short words.
The problem started occurring when I entered a long character string (eg. “asdfghjiklasdfsdfsdkjhdfseds”) in the Special Note field, and then submitted the form. The software stopped responding to user input in this scenario.

The Performance was getting adversely affected when trying to match the regex pattern with the entered long character string.

To find out more about the issue I had faced, I researched online about how regular expressions can affect performance. I found that the performance issue occurs because of  backtracking. I also found  Microsoft’s example of a regex for email address that can cause performance issues. The linked articles provide more detail on backtracking, and on how the issue occurs if the regular expression used is not optimized.

So, when you are required to test a text field that uses a regular expression, always include the following test cases in your testing:

1. Different possibilities of text that matches the regular expression pattern.

2. Different possibilities of text that does not match the regular expression pattern.

3. Different possibilities of text that almost matches the regular expression pattern –  Including, a long string of characters, with no whitespace in between, that nearly (but not completely) matches the regular expression.

Assuming no one will find that bug is a very bad idea!!

Rodney recently posted an article  on incorrect policies of some companies, where they assume that not telling anyone about their security flaws will somehow protect them.
Such companies can not last very long because they incorrectly assume that they are the only intelligent people in the planet.
Someone with malicious intent can always find out your security flaws without you telling him/her. So it’s crucial to remove those flaws instead of trying to hide them.

On a similar note, I want to tell you to never make assumptions about any bug.
Eg. When I am telling you of a server error that occurs in your website, don’t just ignore it by assuming the scenario I told you about will rarely occur. Users are not 100% predictable. No human is. So, your assumption – that only a tester would get such a server error and users would not – is wrong.

Also, if the “rare” bugs you chose to ignore are a lot in number, there is more probability of a user coming across at least some of them. Each bug a user finds has a cumulative effect on driving the user away from you.
If by chance, a user comes across such an error, he/she will be confused and frustrated, and you might lose your audience to someone else who took the time to fix their bugs.

And you wouldn’t want that, would you?

Dilbert Software Quality

Testing Strategy


Here’s the testing strategy given by G.J. Myers in “The Art of Software Testing – 2nd Edition”.

This is a very good strategy and worth following at all times.

1. If the specification contains combinations of input conditions, start with cause-effect graphing.

2. In any event, use boundary-value analysis.

The boundary-value analysis yields a set of supplemental test conditions, but, many or all of these can be incorporated into the cause-effect tests.

3. Identify the valid and invalid equivalence classes for the input and output, and supplement the test cases identified above if necessary.

4. Use the error-guessing technique to add additional test cases.

5. Examine the program’s logic with regard to the set of test cases.

Use the decision-coverage, condition-coverage, decision/condition-coverage, or multiple-condition-coverage criterion (the last being the most complete).

If the coverage criterion has not been met by the test cases identified in the prior four steps, and if meeting the criterion is not impossible (i.e., certain combinations of conditions may be impossible to create because of the nature of the program), add sufficient test cases to cause the criterion to be satisfied.

Black box testing with Cause-Effect Graphs

cause-effect-graph-nodesI’ve been reading “The Art of Software Testing – 2nd Edition” by G.J. Myers. The book introduced me to a concept called Cause-Effect Graphs.

This provoked me to research further about it, and to think how I could implement it into my work.

Below is the summary of what I learnt. Unfortunately, I can not share an example of how I implemented it into my work due to concerns of confidentiality. But I’ll share links with other examples for further understanding of this concept.

Cause-Effect Graphing (CEG) is basically a Black-box testing technique that is used to create test cases according to the specifications provided. (—It is a type of Requirements-Based Testing, also known as Dependency modelling)

CEG can not be used in all scenarios. It can only be used in cases where the test output depends on a combination of test inputs.

For eg, you’re showing your users a set of results based on which one of the 2 options – A or B – they chose while requesting data. The type of output (results) here depend only on one test input (that could either be option A or option B). So, CEG is not possible here .

But, if the results depend on a combination of the selected option (A or B), the current date, the user’s nationality, and the user’s language, CEG could be applied because the output depends upon a combination of these inputs.

What CEG does is that it helps capture the relationships between specific combinations of inputs (causes) and outputs (effects).

—The causes and effects are represented as the  nodes of the graph.
I’m listing down a few advantages of CEG below:
  • —Unlike Boundary Value Analysis and Equivalence Partitioning, Cause-Effect Graphing takes into account input combinations
  • —Helps in selecting, in a systematic way, a set of high-yield test cases out of an astronomical set of input combinations.
  • —As a beneficial side-effect, it also points out incompleteness and ambiguities in the specifications.

Notations used – Nodes

Nodes can have either of 2 states:            0 (absent) or 1 (present)


Notations used – Constraints

Constraints define the relationship between a set of inputs

  • —E => c1 & c2 can not be 1 simultaneouslycause-effect-graph-constraints
  • —I => c1, c2, c3 can not be 0 simultaneously
  • —O => One, and only one,

of c1 & c2 must be 1

  • —R => For c1 to be 1, c2

must be 1

Notations used – “Masks” Constraint

If effect e1 is 1, effect e2 is forced to 0 cause-effect-graph-masks-constraint

Now, that I have introduced you to the building blocks of CEG, let’s go through the method:

  1. Divide the specifications into workable pieces. Trying to work with the whole specs all at once would make the process very, very complicated.
  2. Identify cause & effects in the specifications.Cause = A distinct input condition or equivalence class of input conditionsEffect = An output condition or a system transformation
  3. Analyze the semantic content of the specification & transform it into a Boolean graph linking the cause and effects   (Cause-Effect Graph)
  4. Annotate the graph with constraints describing combinations of causes and/or effects that are impossible due to syntactic/environmental constraints
  5. Convert graph into a limited-entry decision table by methodically tracing state conditions in the graph
      1. Select an effect to be the present state (=1)
      2. From the graph, find all combinations of causes (subject to constraints) that will set this effect to 1.
      3. Create a column in the decision table for each combination of causes.
      4. For each combination determine the states of all other effects and place these in each column
    considerations used when tracing cause effect graph
    CEG trace 1
    CEG trace 2
    The final table will look something like this:
    CEG table
  6. Each column in graph represents a test case. So, the last step would be to derive test cases from the table by converting each column to a separate test case.

To keep the number of test cases to a minimal set of high-yield test cases we can assimilate Boundary-value and Equivalence class in the test cases just derived.

I will also list the testing strategy given in the book when i write my next post.

Below are some links with examples of how CEG can be used to create tests. I hope this post was informative for you.

Chapter on CEG from VTU Learning

Good Postal Regulation example