Save info   Get password
Home Submit your blog Edit Account Rules RSS-Archive Contact


Extreme Telecommuting
2007-08-04 13:41:00
"Ten years ago, there's no way this would have worked. Now there are hardly any barriers."- Anthony PageMany of us spend most of our days trapped in a cubical or windowless office. At times I have enjoyed the opportunity to telecommute from home. I've had some good and bad home offices over the years. I've worked with great views and I've worked in basements. I'm a bit envious of James Bach's new digs.I have the pleasure of working from home one day a week. I look forward to this day because I don't have to deal with traffic, I can work in the comfort of my own home, and I can get work done with fewer interruptions.Earlier this week, I came across a CNN story about telecommuters that don't work from home. These telecommuters work from wherever they want to be. They are working globetrotters. Today's technology makes it possible for many people to work from anywhere in the world. I think we are still some time away from this being an option for many employees. Howeve
Read more: Extreme , Telecommuting , Extreme Telecommuting

For Better or For Worse
2007-07-25 21:06:00
I just stumbled across a 1986 quote from then-president of now defunct Ashton-Tate. I think it has application to automation in software testing."A computer will not make a good manager out of a bad manager. It makes a good manager better faster and a bad manager worse faster."-Ed EsberAll software is automation. Therefore, all software testing involves some level of automation.Automation has potential to do good or bad faster. And, faster is not necessarily better.Let's be smart about how and what we automate. Ben Simo QualityFrog.com Questioning Software℠


Keys to Innovation
2007-07-25 19:52:00
Lee Copeland's CAST keynote address referenced in a previous post was not only about books. Good books was one of the items on Lee's list of eight recent innovations in software testing. Lee's complete list is shown below.Innovation s in Software Testing(Lee Copeland's List)Context-Driven School Testing Specialties Test-First Development Really Good Books Open Source Tools Session-Based Test Management Testing Workshops CertificationI was glad to see most of the items on this list. I am especially happy to see the Context-Driven School and Session-Based Test Management on the list. I believe that these have had a significant impact on software testing and have great potential that has not yet been realized.Tester certification may be an innovation but I don't think its impact has been good. In my opinion, the current certification options are bad. (There was a certification debate hosted by AST at CAST this year. Please take a look at Tim Coulter's review: AST Certification D


Woodpeckers, Pinatas, and Dead Horses
2007-07-12 21:38:00
Here's some short blurbs of a few things I took away from CAST sessions.From Lee Copeland's keynote address:"It's nonsensical to talk about automated tests as if they were automated human testing."Write or speak about something you're knowledgeable and passionate about.Combine things from multiple disciplines.From Harry Robinson's keynote address:Weinberg's Second Law: If Builders Built Buildings The Way Programmers Write Programs, Then The First Woodpecker That Came Along Would Destroy Civilization.From Esther Derby's keynote:To successfully coach someone, they must want to be coached and want to be coached by you.From James Bach's tutorial:Pinata Heuristic: Keep beating at it until the candy comes out. ... and stop once the candy drops.unless ...Dead Horse Heuristic: You may be beating a dead horse.yet beware ...If it is a pinata, don't stop beating at it until the candy drops; but if it is a dead horse, your beating is bringing no value. It can be a challenge to determin
Read more: Horses

Exploratory Scripted Automated Manual Testing
2007-07-12 20:18:00
Exploratory testing is often said to be the opposite of scripted testing. Automated testing is often said to be the opposite of manual testing. Instead of placing testing on one end of the spectrum or the other, I find it good to look at these as opposite ends of a spectrum. I believe good testing contains components spread throughout the spectrum.I think we get into trouble when we apply these labels to “testing” as a whole. I believe all software testing has some aspects that are exploratory, some scripted, some automated*, and some manual. Hopefully all are driven by sapience.Here are some things I remember about one of the best testing projects in my experience:• Organizational testing processes were well documented• Requirements were well documented• We had enforceable quantitative requirements• Tests were well planed• Test plans were well documented• Test cases were well defined and traceable to requirements• Tests were well scripted• Automation was an in
Read more: Testing

Read any good books lately?
2007-07-10 15:00:00
"I've never read a book about software testing."- too many testersIn a CAST keynote address about recent innovations in software testing, Lee Copeland relayed a story about asking all the testers at a large respected financial company about their favorite software testing books. Lee said that every one of the testers said they had never read a book about software testing.Lee compared this to a surgeon informing a patient that they've never read a book about surgery, but not to worry because they are a good surgeon.I too have asked a number of testers about their training to be a tester and have often received responses similar to those reported by Lee.I want to pass on Lee's encouragement to read. Lee also heralded the benefits of applying lessons learned outside technology fields to testing (e.g., philosophy and psychology) to software testing.There was a time that there weren't many testing books from which to choose. This has changed. Today, there are many. There are some go
Read more: lately

Too much testing?
2007-07-08 04:32:00
In a recent blog post, Jason Gorman provides some thoughts about the following question:How much testing is too much?To me, this is like asking "how much cheese would it take to sink a battleship?" There probably is an answer - a real amount of cheese that really would sink a battleship. But very few of us are ever likely to see that amount of cheese in one place in our lifetimes.- Jason Gorman As Jason states, we may never encounter too much testing. However, I believe that we testers often include too much repetition in our testing and miss many bugs that are waiting to be discovered. This becomes especially likely when we limit our testing to scripted testing or put our test plans in freezers. Repeating scripted tests -- whether manual or automated -- is unlikely to find new bugs. To find new bugs, we testers need to step outside the path cleared by previous testing and explore new paths through the subject of our testing.Executing the same tests over and over again is like a grade


I'm a user and I just did that
2007-07-06 20:03:00
Michael Bolton just blogged about the sometimes common exchange between testers and developers that often goes something like this:TESTER: I found this really important bug. Look at this. Let me show you ...DEVELOPER: No user would do that.TESTER: But, I'm a user and I just did that.DEVELOPER: But, the real users won't do that.Michael states that what the developer really means is "No user that I've thought of, and that I like, would do that on purpose." This is very true. Michael also points out that we testers are not the real users and may do things that the real users are not expected to do.Thinking of users that the developer did not think of is an important service we testers provide. This becomes especially important when we put applications on the Internet. We need to consider the users and user behavior that the developers did not consider. I believe it is our responsibility as testers to tactfully provide the development team with the information they need to make an infor


Ugly Babies: Another Reason Why We Need Testers
2007-06-27 19:20:00
... I got on the train ... And I noticed that the woman across from me in the aisle had her baby with her. Ugly baby. Ugly baby.From the other end of the coach comes this guy and he was very drunk and he was staring at the baby. ... And the guy said, "I'm looking at that ugly baby. That's a horrible looking baby lady. Where'd you get that baby from?"And the woman said, "I don't have to take that!" And she snatched the emergency cord and the train came to a screeching halt, and the conductor came running in.Now this was his moment. At this moment he represented the Pennsylvania Railroad. And he said, "what's going on here?"And the woman said, "This man just insulted me. I don't have to spend my money and ride this railroad and be insulted. I'd rather walk."And the conductor said, "calm down! Calm down! Madame there's nothing, nothing that the Pennsylvania Railroad will not do to avoid having situations such as this. Perhaps it would be more to your convenienc
Read more: Babies , Reason

Software Development Life Cycle Explained
2007-06-24 01:42:00
A couple weeks back, there was some discussion on the software-testing group about the use of the term "SDLC" on resumes. (Matt Heusser posted some excerpts from this conversation here.) My warning flags go up when people claim to have a "full understanding" of the SDLC. I sometimes see this as an indication that someone may not be as experienced as they claim. The "SDLC" will vary from one company to another and even one project to another. SDLC is a process documentation term -- and there are many differing processes used to develop software. Its not how people talk in the real world.I recently re-stumbled upon a description of the SDLC that seems to be fairly common across many companies and projects. Perhaps this is what people are referring to when they claim full knowledge of the SDLC. :)Software doesn’t just appear on the shelves by magic. That program shrink-wrapped inside the box along with the indecipherable manual and 12-paragraph disclaimer notice actually came to you by
Read more: Development , Cycle , Software Development , Life Cycle

How many load generators do I need?
2007-06-23 12:55:00
How many load generators do I need to run a [insert number here] user load test on a web application?I am often asked how many load generators are needed for a load test with a certain number of simulated users.My answer: It depends.It depends on the system under test. It depends on your test tool. It depends on your specific script. It depends on your load generation hardware.There is no straight forward answer to this question. There is no formula that can be used to extrapolate an answer. There is no one-size-fits-all rule of thumb. Some tool vendors will attempt to provide an answer, but they are wrong. I once spent a half hour arguing with a tool vendor support representative that claimed that I could run 200, and no more than 200, simulated users per load generator regardless of what those simulated users did or what hardware I used to host them. I had successfully simulated 1200 users with this tool for one script but could not simulate 50 users with another script. The number I


Marching Through Hell
2007-06-21 01:01:00
There is a Churchill quote about when you're marching through hell, just keep marching. Well, that's basically what we're doing. We need to slow this down and get it right.- Charles BurbridgeCFO, Los Angeles Unified School DistrictA couple weeks ago, the Los Angeles Unified School District's new computer system incorrectly paid over 32,000 employees. That's almost a third of the district's employees. And the trouble started this past January.Glitches in Los Angeles Payroll System Spark Furor, Education WeekL.A. Unified payroll snafu's costs soaring, L.A. TimesI've been involved in some software projects with a "hell" phase in the SDLC. I've worked on projects that failed. I've been involved with projects that went millions over budget. However, I'm thankful that I have never worked on a project for which tens of thousands of production issues were reported in a handful of months.Through the replacement of our aging financial, human resources, payroll andprocurement syst


Crapper Automation
2007-06-19 21:37:00
Mechanization best serves mediocrity.- Frank Lloyd WrightAutomation in public restrooms is becoming commonplace. The restrooms at my workplace have recently been remodeled and automated. The lights turn on and off automatically. The toilets flush automatically. Soap dispenses automatically. Faucets turn on and off automatically. Paper towels dispense automatically. All of the above appear to be good candidates for automation. Automation in the restroom is supposed to improve cleanliness, reduce maintenance, and cut supply costs. However, I am not certain that this is true in implementation. I have found the following bugs in our newly automated water closets.I have to walk about ten feet into the restroom in the dark before the lights come on.It has been reported that if one spends too much time seated on the throne, the lights will turn off leaving the occupant to finish their business in the dark.The paper towel dispensers spit out towels when someone walks by the dispenser.T


Installing Windows Vista
2007-06-16 16:42:00
I have not yet made the jump to Windows Vista on any of my personal machines.We have installed Vista on some machines in the test lab where I work. When we started playing with a Vista beta, Vista was not allowed on our network but our customers were using Vista; demanding that we test our products on Vista. There were numerous technical and political battles that had to be fought to get Vista installed in our test lab.We quickly discovered that Microsoft's minimum system requirements are just the minimum to install the OS. I can't imagine any user being happy with Vista on a machine that just met the minimum requirements.We encountered incompatible DVD drives from a major PC manufacturer. We followed Microsoft's instructions for copying the DVD to a hard drive for installation from the hard drive instead of the DVD only to have the Vista install inform us that it cannot be installed as Microsoft instructed.We've had to call Microsoft for permission to reinstall failed installa


Modeling the Windows Calculator: Part 2
2007-06-15 18:28:00
Adding Basic ValidationsIn the previous post, I created a simple model for starting and stopping the Windows calculator, and for switching between standard and scientific modes. I then created the code needed to execute that test and ran a test that hit each of the defined actions once.As the next step, I reran the test with the MBTE configured to capture GUI object information as it executes. This created a checkpoint table that I then ran through a script that removes duplicates and combines rows that are the same for multiple states. I also manually reviewed this table to verify that the reported results are as I expected. I made some tweaks to the table based on my expectations. I can then use this checkpoint table as input for the next test execution. You may view the edited checkpoint file using the link below.The checkpoint table is one of two table formats that I use for defining test oracles. I call the other format a state table. The state tables contain one validatio
Read more: Modeling , Calculator

Modeling The Windows Calculator: Part 1
2007-06-14 19:57:00
I have received a number of requests for some sample models. Based on a question I received a couple weeks ago, I'd like to create a test model for the Windows Calculator . The Windows calculator contains some things that are very simple to model as a state machine (such as switching between standard and scientific modes) and other things that do not have clear distinguishable states (such as performing the actual calculations).I plan to model the calculator a piece at a time in a series of blog posts. I welcome your input.I will start by modeling the obvious states that I see in the Calculator's user interface.At the highest level, I can partition the Calculator's behavior into two states: running or not running. Next, the calculator has two major modes of operation: standard and scientific. After a little experimentation, I see that if I stop the calculator it will return to the previous mode when it is restarted. These transactions can be modeled as follows:calc.standard -> calc.
Read more: Modeling

Sapience In The Age Of Automation
2007-06-12 04:30:00
Society has unwittingly fallen into a machine-centered orientation to life, one that emphasizes the needs of technology over those of people, thereby forcing people into a supporting role, one for which we are most unsuited. Worse, the machine-centered viewpoint compares people to machines and finds us wanting, incapable of precise, repetitive, accurate actions. Although this is a natural comparison, and one that pervades society, it is also a most inappropriate view of people. It emphasizes tasks and activities that we should not be performing and ignores our primary skills and attributes -- activities that are done poorly, if at all, by machines. When we take the machine-centered point of view, we judge things on artificial, mechanical merits. The result is continuing estrangement between humans and machines, continuing and growing frustration with technology and with the pace ans stress of a technologically centered life.- Donald A. NormanThings That Make Us Smart:Defending Human At
Read more: Automation

Performance: Investigate Early, Validate Last
2007-06-10 17:47:00
Performance and load testing is often viewed as something that has to be done late in the development cycle with a goal of validating that performance meets predefined requirements. The problem with this is that fixing performance problems can require major changes to the architecture of a system. When we do performance testing last, it is often too late or too expensive to fix anything.The truth is that performance testing does not need to happen last. Load test scripting is often easier if we wait until the end, but should we sacrifice quality just to make testing easier?Scott Barber divides performance testing requirements and goals into the following three categories:Scalability -- extremely technicalStability -- mostly technicalSpeed -- fuzzy: some technical, other usabilitySpeed is where things get fuzzy. Some speed requirements are quite definable, quantifiable and technical; others are not.- Scott BarberScott says that hard measurable requirements can usually be defined for sca
Read more: Performance , Investigate , Early , Validate

Free Internet: Flushing The Web
2007-06-09 09:45:00
Although we understand that there's a lot of crap on the web, we also believe strongly in providing equal opportunity access to all our users. - Google The folks at Google are working hard to provide innovative services to the world. A couple months ago, they announced the Beta release of their free in-home wireless broadband service: Google TiSP.Google TiSP promises free, fast, and easy to install wireless broadband service. Sorry, this services is only available in the United States and Canada.Check it out here.What is TiSP?Toilet Internet Service Provider When things go wrong with TiSP, they go very, very wrong. Let's leave it at that.Happy Flushing ! Now where'd I put my WiiHelm? Ben Simo QualityFrog.com Questioning Software℠


Bad Messages
2007-06-08 19:30:00
I regularly spend a great deal of time tracking down the root cause of software errors -- both on and off the job. Much of the investigation effort could have been avoided if I were not presented with incomplete or incorrect error messages. The text of error messages appears to be commonly overlooked by software developers and testers.One of my early test automation development tasks was to fix the error messages in a test tool. The tool was used to validate the structure of data exchanged between computer systems. In nearly all cases, this tool displayed an error dialog window stating that the data was not valid. This tool gave the user (a tester) no information about what was wrong with the data. The lack of detail in these messages required that the user manually examine the data -- bit by bit -- with a protocol analyzer. This turned investigation of errors reported by an automation tool into a tedious manual task. The tool encountered a problem but did not report the source of the


Model-Based Test Engine Benefit #4: Generate and execute new tests – and find new bugs
2007-06-08 00:49:00
The last -- and perhaps the best -- major benefit of implementing a Model-Based Test Engine (MBTE) is automation that is capable of generating and executing tests that have not previously been executed manually. Traditional regression test automation simply retraces test steps that have already been performed manually. This may find new bugs that show up in a previously tested path though an application but will not find bugs off the beaten path. In his book "Software Testing Techniques", Boris Beizer compares the eradication of software bugs to the extermination of insects. Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual.- Boris BeizerSoftware Testing TechniquesWhen we apply any method to finding bugs, we will find bugs that are found by that method. However, other bugs will remain. Finding new bugs requires variation in testing, not repeating the same thing over and over. Repeatability is often advertised


The Shopping List
2007-08-21 17:16:00
A couple nights ago, my son and I were in a store with a list of grocery items to purchase. My wife had given the list to our son and he was calling out a few items at a time. If he called out too many items, I'd forget them all and ask him to start over. When he called out something that I knew was nearby, we'd head in that direction and get that item. Darting back and forth may not be a very efficient way to shop, but it likely costs me less than systematically going through the store. When I go grocery shopping with an always-hungry 10 year old boy, it is very likely that we will buy twice as much stuff as is on our list.Although he will claim that I always say "no", my son is skilled at talking his dad into buying things. In this trip to the store, he first asked for some expensive pastries knowing that I would say no -- like I always do. :) Then he proposed the cookies that I think he really wanted and gave me a well-thought-out argument as to why these were better and c
Read more: Shopping

Failure Usability
2007-08-18 11:43:00
One of my pet peeves about software is bad error messages. In my view, a bad error message is one that does not tell the user how the error impacts them and what they need to do in response to the error. Too many messages fail to communicate this information in terms that the software's user is going to understand. Too many of these messages are written for developers, not users.There is a place for error logging in terms that help developers and testers troubleshoot and fix problems. This information is often best written to log files, not displayed in the user interface.Pradeep Soundararajan and I recently discussed some of our experiences with error messages. You can listen to excerpts from this conversation using the link below.Click to listen: ErroneousErrors.mp3After the above conversation, Pradeep walked me through an exercise that demonstrated a case in a popular office application in which I, the user, was not sufficiently informed that an error was occurring. It was ob
Read more: Usability

Excuses, Excuses
2007-08-15 20:18:00
I have heard a variety of responses from developers in response to bugs I report. Some are good, some bad, and some are just plain ugly. Here are a few handfuls.That's strange.How'd you do that?It works on my machine.I already fixed that. You'll have it in the next build.No user would do that.That's not how you're supposed to do that.It's a data problem. Tell the users to fix the data.That's a cool bug! Show me again.I didn't touch that code.It works as designed.It works as coded. [Well, duh. What else would it do?]That's not a bug, it's a feature.I can't test everything.Thank you. ... and my absolute favorite (this came from a development manager)Don't judge it, just test it.The difference between the good and bad is often the relationship between developer and tester. Testers need developers to create something to test. Respect your developers. Communicate with respect and help turn the ugly responses into good responses.As I've heard James Bach say: Testers don'
Read more: Excuses

Things We Know
2007-08-08 18:00:00
I find it at work. I find it in online forums. I find it in books. I find it in papers. I find it in blogs. I find it at conferences.I hear it from experts. I hear it from freshers. I hear it from friends. I hear it from managers. I sometimes even hear it come out of my own mouth.It influences testers. It influences developers. It influences managers that influence testers and developers. It impacts customers.It wastes time. It wastes money. It frustrates developers. It confuses executives. It demeans testers. It decreases quality in the name of improvement.It permeates the practice of developing and testing software.What is this ubiquitous it?It is testing folklore.It ain’t so much the things we don’t know that gets us in trouble. It’s the things we know that ain’t so.- Artemus WardHere are some examples I pulled off the top of my head:There are best practicesTool vendors know those best practicesThe right tools make good testingTesters are the enemies of


Extreme Telecommuting
2007-08-04 13:41:00
"Ten years ago, there's no way this would have worked. Now there are hardly any barriers."- Anthony PageMany of us spend most of our days trapped in a cubical or windowless office. At times I have enjoyed the opportunity to telecommute from home. I've had some good and bad home offices over the years. I've worked with great views and I've worked in basements. I'm a bit envious of James Bach's new digs.I have the pleasure of working from home one day a week. I look forward to this day because I don't have to deal with traffic, I can work in the comfort of my own home, and I can get work done with fewer interruptions.Earlier this week, I came across a CNN story about telecommuters that don't work from home. These telecommuters work from wherever they want to be. They are working globetrotters. Today's technology makes it possible for many people to work from anywhere in the world. I think we are still some time away from this being an option for many employees. Howeve
Read more: Extreme , Telecommuting , Extreme Telecommuting

For Better or For Worse
2007-07-25 21:06:00
I just stumbled across a 1986 quote from then-president of now defunct Ashton-Tate. I think it has application to automation in software testing."A computer will not make a good manager out of a bad manager. It makes a good manager better faster and a bad manager worse faster."-Ed EsberAll software is automation. Therefore, all software testing involves some level of automation.Automation has potential to do good or bad faster. And, faster is not necessarily better.Let's be smart about how and what we automate. Ben Simo QualityFrog.com Questioning Software℠


Keys to Innovation
2007-07-25 19:52:00
Lee Copeland's CAST keynote address referenced in a previous post was not only about books. Good books was one of the items on Lee's list of eight recent innovations in software testing. Lee's complete list is shown below.Innovation s in Software Testing(Lee Copeland's List)Context-Driven School Testing Specialties Test-First Development Really Good Books Open Source Tools Session-Based Test Management Testing Workshops CertificationI was glad to see most of the items on this list. I am especially happy to see the Context-Driven School and Session-Based Test Management on the list. I believe that these have had a significant impact on software testing and have great potential that has not yet been realized.Tester certification may be an innovation but I don't think its impact has been good. In my opinion, the current certification options are bad. (There was a certification debate hosted by AST at CAST this year. Please take a look at Tim Coulter's review: AST Certification D


Woodpeckers, Pinatas, and Dead Horses
2007-07-12 21:38:00
Here's some short blurbs of a few things I took away from CAST sessions.From Lee Copeland's keynote address:"It's nonsensical to talk about automated tests as if they were automated human testing."Write or speak about something you're knowledgeable and passionate about.Combine things from multiple disciplines.From Harry Robinson's keynote address:Weinberg's Second Law: If Builders Built Buildings The Way Programmers Write Programs, Then The First Woodpecker That Came Along Would Destroy Civilization.From Esther Derby's keynote:To successfully coach someone, they must want to be coached and want to be coached by you.From James Bach's tutorial:Pinata Heuristic: Keep beating at it until the candy comes out. ... and stop once the candy drops.unless ...Dead Horse Heuristic: You may be beating a dead horse.yet beware ...If it is a pinata, don't stop beating at it until the candy drops; but if it is a dead horse, your beating is bringing no value. It can be a challenge to determin
Read more: Horses

Exploratory Scripted Automated Manual Testing
2007-07-12 20:18:00
Exploratory testing is often said to be the opposite of scripted testing. Automated testing is often said to be the opposite of manual testing. Instead of placing testing on one end of the spectrum or the other, I find it good to look at these as opposite ends of a spectrum. I believe good testing contains components spread throughout the spectrum.I think we get into trouble when we apply these labels to “testing” as a whole. I believe all software testing has some aspects that are exploratory, some scripted, some automated*, and some manual. Hopefully all are driven by sapience.Here are some things I remember about one of the best testing projects in my experience:• Organizational testing processes were well documented• Requirements were well documented• We had enforceable quantitative requirements• Tests were well planed• Test plans were well documented• Test cases were well defined and traceable to requirements• Tests were well scripted• Automation was an in
Read more: Testing

Page 3 of 5 « < 2 3 4 5 > »
eXTReMe Tracker