A question, answers and a session with James Bach..

Huh!! I still can’t believe I had attended an online training session from James Bach.. It was great talking to a genius who breaths Testing πŸ™‚ I would like to thank James for the time once again..

Now, let’s come to the point…

Last week, I was preparing a document – a product research document to be precise. I had to go through our competitor sites and pick out products that are not offered by our service. And make a list comparing the features of the competitor products and our products. I was left wondering what can a Tester learn from this exercise.. I was very frustrated with “not” testing our application and was not interested in browsing through other sites “peeking” at what they had to offer..

It was at this time I saw Pradeep Soundararajan online in skype. So I thought of asking him about this whole exercise and what can I learn from this scenario. Pradeep replied: (I’m copy pasting the reply without editing, don’t want to spoil my readers expectations :))

That’s one of the fantastic questions I have heard. Not everything that we do might result to a learning. We must be aware that some learning is conscious and some are sub conscious. At times the sub conscious becomes conscious and we say, “Ah! I have seen something like this before”.

While you are exploring competitors products, you might also want to test a little bit. Or, you could also try to make notes of how you search, what is your thought process when you did whatever you did, what were your alternate ideas and so on, to make a conscious effort to learn something conscious.

Oh my god! This is certainly more than what I thought!!

Lessons learned:

  1. Make a conscious effort to learn something
  2. Make notes on how you go about searching
  3. Track your thought process when you are doing something
  4. Try to device new/alternate ideas in doing the exercise

Thanks a lot Pradeep πŸ™‚

And now comes the bigger surprise.. 2 days later, I logged into Skype and saw the one and only James Bach online πŸ™‚ As you might have figured out now (through the link above), he is offering testing lessons through Skype. Though I was satisfied with Pradeep’s answer,Β  just wanted to know what James thought of my “Product Research”. So I asked him the same question. And I was struck with lightning when I saw his reply:

I study competing similar products in order to learn specific kinds of things that help me test better:
  • I might discover something about testability by seeing a feature the competitors have that makes their product easier to observe or control
  • I might discover something about risks by studying complaints about competing products
  • I might discover what the general standard of quality is in this product segment
  • I might learn what are considered standard features
  • I might learn about users

For instance: I’m looking at medical devices that have been recalled,so that I can better argue for why certain kinds of testing is needed for the medical device I am working on now. Pay special attention to: error messages, input methods, sample data, claims made about compatibility etc. This knowledge helps, when you are arguing about which bugs are important and which are not. If you can point to other products that work better, you get leverage

I shared Pradeep’s answer with him. And I was offered a little more wisdom πŸ™‚

“He’s right that you pick up things that you don’t necessarily know you will need. Imagine that you are invited to a bug triage meeting. Someone might say “why do we need to fix this bug? People can work around it.” Perhaps you will say, “Our biggest competitor doesn’t have this feature. If we do this well, we might gain market share.” or “Our biggest competitor has this feature, but it’s unreliable. If we can do it a lot better, we can gain market share.” Whatever people are saying against your competitors, make sure they can’t say that against you.”

I am short on words by this time.. πŸ™‚

And it did not end there.. He shared 2 XLS documents which contained some data about software recalls and wanted me to find patters in it – that’s not all, each pattern should be described with a label which should NOT be more than four WORDS.

By this time I was totally out of my mind and misread the “four WORDS” as “four LETTERS” (BIG MISTAKE!). But I am happy that I misread it – you know why? Because he gave me one terrific example of reporting the summary of a bug πŸ™‚

Here they are:

Platform Incompatibility: The product fails in some cases to function with a third-party component that it depends upon.

we can turn this into two, perhaps;

Transient Platform Incompatibility: The product fails in some special cases to function with a third-party component that it depends upon.

Chronic Platform Incompatibility: The product fails always to function with a certain optional third-party component that the customer may be using.

I always try to report a bug with as much details as possible – with proper Summary too. But since I misread James’ words, I ended up pathetically short on this one. Do you want to see the pattern I gave to James? (Now, don’t laugh ok? It was just a small mistake :))

“SYST – failed by compatibility” !!!! 😦

Now that I understood what James meant, I would try to rewrite one of my patterns – “DEP – failed by dependence with other systems”. (Readers please excuse, this is totally related to the XLS documents James provided me – I cannot share it with you without his permission)

“Dependency with product module – Two models installed with Version 8 software is showing a potential risk of over infusion related to a warning message in the user interface, when used in conjunction with the Product Module “.

(I feel this is much better than my earlier crap reporting, but I would like to know what my readers think about this..)

So, the lessons learned:

  1. Even if you are not testing, you could always look for testability
  2. You can get information about basic standards, quality of a particular type of product while browsing
  3. You could get possible bugs by going through product recalls and check whether any of those can be reproduced in your products
  4. When you are listening to something, concentrate on the topic πŸ™‚
  5. Ask questions if something is not clear to you (I didn’t, at least for this time!)
  6. Always try to report an issue with precise details – so that developers can reproduce and fix it

Dear James, I hope I have not disappointed you πŸ™‚ I would like to attend many more sessions from you..

Readers, sorry for the lengthy post! Please share your thoughts on my “Question” and the answers of Pradeep and James.

10 thoughts on “A question, answers and a session with James Bach..

  1. I feel I’ve made a very good choice by helping you.

    Now I understand why your descriptions were so terse! πŸ™‚

    The purpose of looking for patterns is to identify heuristics that may be used to justify testing and find new bugs. I think your latest attempt to specify the pattern is now too verbose and specific for that. Can you see a more universal pattern there that doesn’t have to do with “version 8” and “infusion pumps”?

    Also, the data I gave you is all public. I got it by searching the FDA recall database using the word “software”.

    Good job.

    — James

    • Dear James,
      Thank you for your time and valuable comment πŸ™‚ I would be keeping that in mind and will try to improve – lesson learned: Keep your bug summary reports simple πŸ™‚

  2. Pradeep and James, 2 gems i have always admired and i have always been a silent reader of their blogs, trying to improvise my skills.
    The best part of James reply according to me was
    ” Whatever people are saying against your competitors, make sure they can’t say that against you.”
    This is an approach which makes some people different than from others. James approach and his thoughts have a great meaning in them from which many lessons can be learnt in Testing.


  3. Studying competitors products is an excellent testing exercise to me. James’ approach demonstrated a typical “diverse” thinking that is required for a tester. Given a problem in how many different (often contrasting) approaches/paths one can take to attack the problem.

    James’ work and works of thinkers like Jerry Weinberg (and his “Introduction to general systems thinking”), Fritjof Capra (Turning point and Tao of Physics), Nicholas Taleb (Black Swan) – helped me to broaden my thinking and deploy a rich set of strategies for a given (testing) problem.

    Most recently, reading “Outliers” by Malcolm Gladwel – the concept of “diverse” thinking is becoming strong in me. Seeing patterns and connections between entities in a systems and systems to other systems is a key tester skill that very few understand and work towards it.

    By getting trained by James – you are on the right route … keep it up

    Shrini Kulkarni

  4. You know what … a single important lesson (to me) from this post and James’ views – When looking for ideas for comparing two market products or studying strengths of a product to be releases in market – Look for “product recall” as a “guide-word heuristic” Like SFDPOT .. (I hope you know this acronym).

    A guide word heuristic reminds you something that would be useful for a given moment which you can expand and deploy a larger and richer set of ideas.

    “Product recall list” or “List of publically known problems” – reminds you to look for specific dimension and hence opens up a new world of ideas to build on.

    So it is not about keeping “bug report small, simple and concise”. James is hinting you recognize the heuristic that is at play and generalize it.

    BTW working from a generic idea to specific and specific ideas to generic (two extreme polarities) help you to broaden/sharpen your thinking.

    Can you think like an eagle at one instance and start seeing through microscope and work with micro organism? That is broad spectrum you need to develop.


  5. Dear Shrini,

    Thank you for your time and insightful comments. I’m happy that the testing “gurus” I have always admired is pouring in with pearls of wisdom πŸ™‚ I have not read the books mentioned by you, but I will surely try to get a copy of them. I’ve not attended any workshops on testing, but I’m familiar with the SFDPOT heuristic – but frankly, I am not so sure how I can employ that in my testing 😦 I’m hoping to attend one of those Rapid Software Testing workshops and sharpen my skills. I know there is still a long way to go in learning the craft – and I’m doing my best. πŸ™‚

    Thanks once again for your detailed comments..

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s