Saturday, August 20, 2011

I am in love with you!

Lost in your steps - I realized the severity, tried hard to reproduce but you were developed by someone else. What could I do?

I cannot remove you from my memory, you have been saved as a known issue. Just wondering if I should let her know about you because she is very tough on those who bug me... I want you back in my life... I want you to get fixed and come clean to me...

I searched for you in my entire system but you hid so well... I cried & cried, fought for you in the bug triage, went back to my old system to retrace our sweet memories but no use... You taught me the value of patience and observation. I stayed awake whole night waiting for the one call, one message but your new partner took away all your attention. I am a stranger to you now...

Remember the first time I saw you, the first screenshot I took, the first letter I wrote to your developer?
Why me dear? Why me???? Come back please!

I'm sorry dear. I should not have shouted in front of everyone. I realize that mistakes happen by everybody. Next time, I will not stress your system, I will not make you run so fast, I will give enough memory, enough hard disk, but give me just one more chance!

I can't spend my weekend without you. If you want, I am ok to meet all your relatives. I am spending time reading our old conversation logs, the tickets we bought together,,, please onde ondu salla plz (Please Only chance please).

thank you so much friends. She came back to me this morning. Her developer fixed her thoughts on me & made me understand the importance of Dev relations. She is going to introduce me to her relatives today. Nervous & excited at the same time. Its going to be a great weekend! :)

Leia Mais…

Tuesday, August 16, 2011

CAST 2011 - Tester Competition - Miagi-Do School Team

After Part I and Part II , here is the blog post about the CAST2011 Tester competition.
On Day 1 - Aug 08th evening, there was a testing competition for four hours from 06:00 PM to 10:00 PM US time. After the awesome sessions the first day, it was time for some fun. And how do testers have fun? Testers have fun by competing in a testing competition.

As I belong to the Miagi-Do School, all the students of Miagi-Do formed one team. There were few other testers who were not part of the school also joining us. So, the team members were: Matthew Heusser, Markus Gaertner, Michael Larsen, Adam Yuret, Elena Houser, Justin Hunter, Simon Schrijver and myself. We were never in chance of winning the cash prize of $1401 as Matthew Heusser was one of the organizers. And still we wanted to participate in this contest. We wanted to test as the Miagi-Do school team!

06:00 PM - Setting up and Learning the Application
As we had more Mac machines than Windows and the application to test was a Windows app, we paired. Michael & Matt, Markus & Adam, Justin & Simon, Elena & myself formed different pairs. The first half an hour was very irritating. The app was close to 19 MB and the network speed was too slow compared to the usual speed. Close to 50-60 testers were accessing the same link from the same hall! I and Elena had USB and we got the app downloaded on them and passed to others in our team. By that time, Michael had already downloaded the app. Then we started to play around, learn what the whole app is about, read the document which came along with the installer. Some clicks here & there, there were few Access Violation errors displayed.

06:45 PM - Calm planning amidst chaos 
Markus takes the lead and brings some calm amidst the chaos. Justin points that instead of everyone testing in their own way, we could have a plan of attack. Markus brings the flip charts to play. After our initial testing, we set ourselves a time limit of 20 minutes of testing sessions followed by 5 minutes of de-brief. The first few sessions were interesting - everyone took different features and going was smooth. The bugs were found and it was time to log them.

07:30 PM - A bit disconnected but focussed on testing
I can't speak for other pairs but our pair (Elena & myself) found some cool Access Violation errors. By this time, we found 3-4 errors. They were not easily reproducible but popped up now & then. After spending close to 40 minutes, we wanted to nail the errors. We also went to the developer who was also in the same room and asked him if he needed exact steps or just the error logs were sufficient. He was happy wit the error logs and the screenshots.

08:45 PM: Bug reporting in full flow
By this time, we had captured most of the bugs and we decided to log them. We had to upload the screenshots, video and the error logs. All these had to be zipped and uploaded. We tried uploading them to the specified location but it failed twice. Sometimes it displayed 90% uploading and then failed. Then I thought of uploading them to my most trusted friend - Dropbox. All the zip files were dropped onto my account and the link was shared. We felt happy that we overcame the link barrier.

09:30 PM: Test Report & Bug Hunting
Matt, Markus, Justin & Adam were busy with preparing the final test report. Matt asked me to report the bugs on the document too from which he would consolidate all the bugs in the final report. We were busy logging bugs and did not update the document. In fact when the team was busy adding sections to the final report, Elena & I were still hunting bugs and reporting them.

What did I learn:
This team rocked! Everyone took a considerable amount of tasks out of the final list of tasks and contributed equally to the final test report. The testing coverage in terms of the different features model was also very good. During the de-brief, each pair picked up something new. Sometimes, if they felt that a particular area needed more testing, they continued. The approach was fantastic - 20 minutes of testing followed by 5 minutes of debrief was very effective. With such a big application to test, that was a very good approach to gain coverage and reduce redundant testing.

I liked the way Markus used flip charts to record during de-brief. The flip charts helped a great deal at the end as we were the only team with good flip charts highlighting test coverage. Those also helped in the preparation of the final test report. I liked how Matt organized the final test report. He gave us simple instructions  - update the and he would take over. I liked how Michael picked up the areas left by others for further investigation and filled the gaps. Markus was too good organizing the entire de-brief. Justin was cool with the mind map. Simon, Adam helped with the final test report. Elena was very helpful in paired testing especially the bug investigation for the Access Violation errors. She helped me use Jing (which I had never used before). Me & Elena were also talking with the developer asking if some of the bugs we found were useful for him.

The four hours were very exhausting. It was like an extended weekendtesting session. Being used to the weekendtesting sessions, I was kind of used of competing in these kinds of hectic testing environments. I learnt a lot about team co-ordination, organizing testing & de-brief sessions. I learnt how to prepare a test report for such a project quickly.

And finally, the next day - Matthew Heusser promoted Elena Houser, Michael Larsen and myself to the next level - Black Belt in the Miagi-Do School of Software Testing. That was a proud moment. I am waiting for the next testing sessions to demonstrate the learnings from this tester competition.

Leia Mais…

Sunday, August 14, 2011

CAST 2011 - First Impressions

Preparation for CAST 2011 blog post is here.
Continued here...

Aug 07th  09:00 pm US time: 
I finally reached Seattle after a long flight journey. I was tired. I wanted to go to Courtyard Marriott hotel and on inquiring, I found that it would cost close to 200 USD for a cab. I wanted to board Shuttle Express as per Lanette Creamer. And the cost was just 26 USD. After a journey close to 1 hr, I reached the hotel.
At the hotel lobby, I found Matt Heusser, Lynn McKee and Nancy Kelln. I was sharing the room with Michael Larsen. When I went to the room, he was awake even though it was close to 10:30 pm US time.

Aug 08th  - Friends meet!
Michael & me went down to the hotel lobby and we met Markus Gaertner, Michael Bolton, Matthew Heusser and few other testers ready to walk to the Lynnwood Convention Center. After close to five minutes of walking, we reached the hall. We were at the registration desk of CAST 2011. I saw Jon Bach talking with  a tester. The moment had arrived. I shook hands with Jon and proceeded to the registration desk. I collected the T shirt, conference kit from Dawn Haynes and saw Sajjadul Hakim busy with his laptop.

I saw James Bach, Anne-Marie Charrett, Adam Yuret, Lanette Creamer, Carsten Feilberg, Simon Schrijver, Fiona Charles and Pete Walen. I was so happy. As Jon Bach said: "It felt like a big party". All the online friends I had were in the same hall as me. The day started with Jon, James welcoming everyone and Michael Bolton's keynote. There were no slides as there was some problem with the laptop/projector combination. I remembered my dream :)

Michael Bolton's Keynote
A keynote without any slides. Michael once again highlighted the CBC show - Ideas and the 24 part series of "How to think about Science"
Notable quotes were:

  • Decisions about quality are political and emotional.
  • Shake the constraints. Refuse to accept the apparent constraints.
  • Testers know that things can be different.
What if there is no time to find all bugs?
We find the important bugs.
What if there is no time for that?
We would conduct a Risk-Analysis.
What if its wrong?
We also take an approach which considers non-risky areas. We will learn from mistakes.

Do we accept failures? No, we accept reality!

We testers discover how the product actually works. We do not confirm that it works.

Acceptance of the reality is maturity. We need to remain skeptical [Rejection of certainty]
Michael also highlighted that testing should focus on human values. The document by Dr. Cem Kaner can be found here. Testing as social science provides partial answers which might be useful.

Michael recommended the study of Philosophy, Epistemology, measurements. Testers should be able to tell both the testing story and product story. We need to be beware of bogus metrics. Also, testers need to be aware of Safety language. There is a difference between "It seems" and "This is". It is important to recognize the difference between observation & inference. "The vs A" problem was also discussed. "The terrible problem" sounds so different from "A terrible problem". Heuristics play an important role in a tester's life. We need to learn & unlearn rapidly. Every minute of the keynote was filled with great learning moments!

Troubleshooting Skills by Chris Blain
After the keynote, there were few RED cards flashing across the hall. There were K cards being used to facilitate the discussion. My cards were numbered 186. After the keynote, it was time for some fruits and the track sessions. I attended the 'Troubleshooting skills' session by Chris Blain. I did not attend Markus' session as I had already seen it at EuroSTAR 2010. Troubleshooting skills are important specially when you do bug advocacy. Chris Blain pointed that as testers, we should not take anyone's words for granted. We should work on improving our skills by learning a range of systems. Systems thinking, modeling, thinking from different perspectives and asking lots of questions to gain information might help troubleshoot better.

Multiple tools like the Process Explorer, Visual Studio, WinDBG, Registry Viewer, DTrace, WireShark, Fiddler, ChainSaw, Log Parsers were highlighted. Some interesting stories like the accidental shaking of a wire causing static electricity and a bug was narrated. When I asked about the books, he highlighted the 'Advanced Windows Debugging' by John Robbins. Chris asked us to be aware of the cultural differences too in troubleshooting.

Weekend Testing presentation by me
After lunch, I presented about Weekend Testing As I had dreamed, there was no laptop and I had to borrow James Bach's laptop. Though I logged in to dropbox and clicked on download against the presentation, it failed to download. I soon realized that creating a rule made the download possible. All set, I entered the room. There were close to 15 members in the audience. I was happy to have James Bach, Michael Bolton, Jon Bach, Michael Hunter and Michael Larsen along with Elena Houser in the audience. Every time I present about Weekend Testing, some titbits which were never shared before come to light. After close to 40 minutes, many questions were asked. One of the question was what is required to participate in the session. I told that as long as a tester has passion, skype and knows english, he can participate!

Jon Bach highlighted how weekend testers tested eBay and helped him gain more information to his questions.      He facilitated few sessions with missions on eBay and those were really useful sessions to the weekend testers. Michael Larsen, Co-Founder of Weekend Testers Americas highlighted how weekend testing was a different experience from a facilitator's perspective. Jon, James, Michael liked my presentation style. I was happy to present at CAST :)

Presenting a Compelling Testing Story by Benjamin Kelly
Ben started with a video of a movie and how a junior finally convinced his senior to agree to his views.
"Give compelling reason to take specific action" was the lesson.
He narrated his story where he and his team presented a 15 page report to the senior management. Whenever the management asked if there was any problem, he replied that nothing more than what was present in the report. The Green flags of the project slowly turned to Red. When management was shocked, Ben was even more shocked. The reports were never read! Ben realized that he had used a wrong medium and a wrong format to present the report. The manager was not willing to spend so much time reading the 15 page report.

People do not respect your testing if they don't like your test reporting. As testers, we need to present the test reports in formats which is appreciated by the stakeholders. He highlighted the different ways how we format information - verbal, written & visual. Ben started with Modeling. Asking questions helps gather information and understand what stakeholders want.
Questions like - What would describe project success, what should happen to achieve project success, what might look like failure, what are the most important aspects of the project to each stakeholder help bring every stakeholder to a single page.

Meta modeling - model of model was also highlighted. As testers, we need to compare our meta models to the models of the stakeholders. What is different, what they don't know and learn what they tell you?
Once we learn about the different models, Ben moved to Reporting part!

The two types - Push Reporting & Pull Reporting was described. Mike Kelly's MCOASTER heuristic was also recommended for effective test reporting.
Pull reporting is when someone asks you for a report. If you are not comfortable, ask for time, Ask for clarification if it helps. There is no point in giving a wrong report without understanding the exact questions.
Push Reporting is when you provide information. The most important aspect is that the audience is briefed. Ben suggests that you as a tester take time to ensure that your audience is briefed. Examples are bug reports, critical bug discovery and risk report.

Prepare, know the purpose, have the story, know the model of audience, understand the difference between your model & the audience's model. Also, pay attention to the audience's model of you. Speak the language of the audience!

The next blog post is about the CAST 2011 tester competition.

Leia Mais…

Preparation for CAST 2011

Dec 2010: I was happy when James Bach asked me on Skype if I was interested to be present at CAST 2011. I asked for 1-2 days time and accepted the invitation. I got a speaking slot and Weekend Testing was the topic James wanted me to present on.

I started saving money for this trip. Though the conference fees was waived off, I had to cover the travel, accommodation costs. I knew that it is worth spending 2500 USD for CAST 2011. My mother and few friends thought that I was crazy.

June 2011: I applied for VISA and there were no dates available for VISA interview at Chennai. I had to wait for few more days. The date slots opened and I selected the first date available - July 13th. I had to be in US by Aug 07th and the visa interview was scheduled on July 13th. All the forms were filled & submitted.

July 13th 2011: Visa interview
Few questions were asked about the conference, my company, why my company i snot sponsoring, why am I spending so much, when will I return and so on. Then why did I go to Vietnam & Germany were also asked. I was granted visa. I was very happy. Finally, I could meet the Bach brothers and my first hero - Matthew Heusser.

Aug first week:
I polished the slides, added/deleted few slides and sent the dropbox link of my presentation to James & Jon.

Aug 05th dream:
I had the dream that just before my presentation, the slides were corrupted. James, Cem told me that such things happen and asked me to download again. I accessed dropbox website and found a totally new GUI. Quickly, thanks to some exploration - I found the slides and downloaded them. In between James asked if I really needed the slides for my presentation. I replied with a big NO. I woke up smiling.

Aug 05th 2011 will be a special day. Reason: I got the award: "The Bach Brothers Legion of Testing Merit"

Aug 07th Flight
The flight was at 04:00 am IST from Bangalore to Dubai. I left home at 12:05 am on Aug 07th and reached airport by 1 am IST. Very sleepy, getting ready for the long flight journey.

Leia Mais…