Save info   Get password
Home Submit your blog Edit Account Rules RSS-Archive Contact
Show All




Web Catalog F305 C905 webbutveckling Article Directory




Read any good books lately?
2007-07-10 15:00:00
"I've never read a book about software testing."- too many testersIn a CAST keynote address about recent innovations in software testing, Lee Copeland relayed a story about asking all the testers at a large respected financial company about their favorite software testing books. Lee said that every one of the testers said they had never read a book about software testing.Lee compared this to a surgeon informing a patient that they've never read a book about surgery, but not to worry because they are a good surgeon.I too have asked a number of testers about their training to be a tester and have often received responses similar to those reported by Lee.I want to pass on Lee's encouragement to read. Lee also heralded the benefits of applying lessons learned outside technology fields to testing (e.g., philosophy and psychology) to software testing.There was a time that there weren't many testing books from which to choose. This has changed. Today, there are many. There are some go
Read more: lately

Too much testing?
2007-07-08 04:32:00
In a recent blog post, Jason Gorman provides some thoughts about the following question:How much testing is too much?To me, this is like asking "how much cheese would it take to sink a battleship?" There probably is an answer - a real amount of cheese that really would sink a battleship. But very few of us are ever likely to see that amount of cheese in one place in our lifetimes.- Jason Gorman As Jason states, we may never encounter too much testing. However, I believe that we testers often include too much repetition in our testing and miss many bugs that are waiting to be discovered. This becomes especially likely when we limit our testing to scripted testing or put our test plans in freezers. Repeating scripted tests -- whether manual or automated -- is unlikely to find new bugs. To find new bugs, we testers need to step outside the path cleared by previous testing and explore new paths through the subject of our testing.Executing the same tests over and over again is like a grade


I'm a user and I just did that
2007-07-06 20:03:00
Michael Bolton just blogged about the sometimes common exchange between testers and developers that often goes something like this:TESTER: I found this really important bug. Look at this. Let me show you ...DEVELOPER: No user would do that.TESTER: But, I'm a user and I just did that.DEVELOPER: But, the real users won't do that.Michael states that what the developer really means is "No user that I've thought of, and that I like, would do that on purpose." This is very true. Michael also points out that we testers are not the real users and may do things that the real users are not expected to do.Thinking of users that the developer did not think of is an important service we testers provide. This becomes especially important when we put applications on the Internet. We need to consider the users and user behavior that the developers did not consider. I believe it is our responsibility as testers to tactfully provide the development team with the information they need to make an infor


Ugly Babies: Another Reason Why We Need Testers
2007-06-27 19:20:00
... I got on the train ... And I noticed that the woman across from me in the aisle had her baby with her. Ugly baby. Ugly baby.From the other end of the coach comes this guy and he was very drunk and he was staring at the baby. ... And the guy said, "I'm looking at that ugly baby. That's a horrible looking baby lady. Where'd you get that baby from?"And the woman said, "I don't have to take that!" And she snatched the emergency cord and the train came to a screeching halt, and the conductor came running in.Now this was his moment. At this moment he represented the Pennsylvania Railroad. And he said, "what's going on here?"And the woman said, "This man just insulted me. I don't have to spend my money and ride this railroad and be insulted. I'd rather walk."And the conductor said, "calm down! Calm down! Madame there's nothing, nothing that the Pennsylvania Railroad will not do to avoid having situations such as this. Perhaps it would be more to your convenienc
Read more: Babies , Reason

Software Development Life Cycle Explained
2007-06-24 01:42:00
A couple weeks back, there was some discussion on the software-testing group about the use of the term "SDLC" on resumes. (Matt Heusser posted some excerpts from this conversation here.) My warning flags go up when people claim to have a "full understanding" of the SDLC. I sometimes see this as an indication that someone may not be as experienced as they claim. The "SDLC" will vary from one company to another and even one project to another. SDLC is a process documentation term -- and there are many differing processes used to develop software. Its not how people talk in the real world.I recently re-stumbled upon a description of the SDLC that seems to be fairly common across many companies and projects. Perhaps this is what people are referring to when they claim full knowledge of the SDLC. :)Software doesn’t just appear on the shelves by magic. That program shrink-wrapped inside the box along with the indecipherable manual and 12-paragraph disclaimer notice actually came to you by
Read more: Development , Cycle , Software Development , Life Cycle

How many load generators do I need?
2007-06-23 12:55:00
How many load generators do I need to run a [insert number here] user load test on a web application?I am often asked how many load generators are needed for a load test with a certain number of simulated users.My answer: It depends.It depends on the system under test. It depends on your test tool. It depends on your specific script. It depends on your load generation hardware.There is no straight forward answer to this question. There is no formula that can be used to extrapolate an answer. There is no one-size-fits-all rule of thumb. Some tool vendors will attempt to provide an answer, but they are wrong. I once spent a half hour arguing with a tool vendor support representative that claimed that I could run 200, and no more than 200, simulated users per load generator regardless of what those simulated users did or what hardware I used to host them. I had successfully simulated 1200 users with this tool for one script but could not simulate 50 users with another script. The number I


Marching Through Hell
2007-06-21 01:01:00
There is a Churchill quote about when you're marching through hell, just keep marching. Well, that's basically what we're doing. We need to slow this down and get it right.- Charles BurbridgeCFO, Los Angeles Unified School DistrictA couple weeks ago, the Los Angeles Unified School District's new computer system incorrectly paid over 32,000 employees. That's almost a third of the district's employees. And the trouble started this past January.Glitches in Los Angeles Payroll System Spark Furor, Education WeekL.A. Unified payroll snafu's costs soaring, L.A. TimesI've been involved in some software projects with a "hell" phase in the SDLC. I've worked on projects that failed. I've been involved with projects that went millions over budget. However, I'm thankful that I have never worked on a project for which tens of thousands of production issues were reported in a handful of months.Through the replacement of our aging financial, human resources, payroll andprocurement syst


Crapper Automation
2007-06-19 21:37:00
Mechanization best serves mediocrity.- Frank Lloyd WrightAutomation in public restrooms is becoming commonplace. The restrooms at my workplace have recently been remodeled and automated. The lights turn on and off automatically. The toilets flush automatically. Soap dispenses automatically. Faucets turn on and off automatically. Paper towels dispense automatically. All of the above appear to be good candidates for automation. Automation in the restroom is supposed to improve cleanliness, reduce maintenance, and cut supply costs. However, I am not certain that this is true in implementation. I have found the following bugs in our newly automated water closets.I have to walk about ten feet into the restroom in the dark before the lights come on.It has been reported that if one spends too much time seated on the throne, the lights will turn off leaving the occupant to finish their business in the dark.The paper towel dispensers spit out towels when someone walks by the dispenser.T


Installing Windows Vista
2007-06-16 16:42:00
I have not yet made the jump to Windows Vista on any of my personal machines.We have installed Vista on some machines in the test lab where I work. When we started playing with a Vista beta, Vista was not allowed on our network but our customers were using Vista; demanding that we test our products on Vista. There were numerous technical and political battles that had to be fought to get Vista installed in our test lab.We quickly discovered that Microsoft's minimum system requirements are just the minimum to install the OS. I can't imagine any user being happy with Vista on a machine that just met the minimum requirements.We encountered incompatible DVD drives from a major PC manufacturer. We followed Microsoft's instructions for copying the DVD to a hard drive for installation from the hard drive instead of the DVD only to have the Vista install inform us that it cannot be installed as Microsoft instructed.We've had to call Microsoft for permission to reinstall failed installa


Modeling the Windows Calculator: Part 2
2007-06-15 18:28:00
Adding Basic ValidationsIn the previous post, I created a simple model for starting and stopping the Windows calculator, and for switching between standard and scientific modes. I then created the code needed to execute that test and ran a test that hit each of the defined actions once.As the next step, I reran the test with the MBTE configured to capture GUI object information as it executes. This created a checkpoint table that I then ran through a script that removes duplicates and combines rows that are the same for multiple states. I also manually reviewed this table to verify that the reported results are as I expected. I made some tweaks to the table based on my expectations. I can then use this checkpoint table as input for the next test execution. You may view the edited checkpoint file using the link below.The checkpoint table is one of two table formats that I use for defining test oracles. I call the other format a state table. The state tables contain one validatio
Read more: Modeling , Calculator

Modeling The Windows Calculator: Part 1
2007-06-14 19:57:00
I have received a number of requests for some sample models. Based on a question I received a couple weeks ago, I'd like to create a test model for the Windows Calculator . The Windows calculator contains some things that are very simple to model as a state machine (such as switching between standard and scientific modes) and other things that do not have clear distinguishable states (such as performing the actual calculations).I plan to model the calculator a piece at a time in a series of blog posts. I welcome your input.I will start by modeling the obvious states that I see in the Calculator's user interface.At the highest level, I can partition the Calculator's behavior into two states: running or not running. Next, the calculator has two major modes of operation: standard and scientific. After a little experimentation, I see that if I stop the calculator it will return to the previous mode when it is restarted. These transactions can be modeled as follows:calc.standard -> calc.
Read more: Modeling

Sapience In The Age Of Automation
2007-06-12 04:30:00
Society has unwittingly fallen into a machine-centered orientation to life, one that emphasizes the needs of technology over those of people, thereby forcing people into a supporting role, one for which we are most unsuited. Worse, the machine-centered viewpoint compares people to machines and finds us wanting, incapable of precise, repetitive, accurate actions. Although this is a natural comparison, and one that pervades society, it is also a most inappropriate view of people. It emphasizes tasks and activities that we should not be performing and ignores our primary skills and attributes -- activities that are done poorly, if at all, by machines. When we take the machine-centered point of view, we judge things on artificial, mechanical merits. The result is continuing estrangement between humans and machines, continuing and growing frustration with technology and with the pace ans stress of a technologically centered life.- Donald A. NormanThings That Make Us Smart:Defending Human At
Read more: Automation

Performance: Investigate Early, Validate Last
2007-06-10 17:47:00
Performance and load testing is often viewed as something that has to be done late in the development cycle with a goal of validating that performance meets predefined requirements. The problem with this is that fixing performance problems can require major changes to the architecture of a system. When we do performance testing last, it is often too late or too expensive to fix anything.The truth is that performance testing does not need to happen last. Load test scripting is often easier if we wait until the end, but should we sacrifice quality just to make testing easier?Scott Barber divides performance testing requirements and goals into the following three categories:Scalability -- extremely technicalStability -- mostly technicalSpeed -- fuzzy: some technical, other usabilitySpeed is where things get fuzzy. Some speed requirements are quite definable, quantifiable and technical; others are not.- Scott BarberScott says that hard measurable requirements can usually be defined for sca
Read more: Performance , Investigate , Early , Validate

Free Internet: Flushing The Web
2007-06-09 09:45:00
Although we understand that there's a lot of crap on the web, we also believe strongly in providing equal opportunity access to all our users. - Google The folks at Google are working hard to provide innovative services to the world. A couple months ago, they announced the Beta release of their free in-home wireless broadband service: Google TiSP.Google TiSP promises free, fast, and easy to install wireless broadband service. Sorry, this services is only available in the United States and Canada.Check it out here.What is TiSP?Toilet Internet Service Provider When things go wrong with TiSP, they go very, very wrong. Let's leave it at that.Happy Flushing ! Now where'd I put my WiiHelm? Ben Simo QualityFrog.com Questioning Software℠
Read more: Free Internet

Bad Messages
2007-06-08 19:30:00
I regularly spend a great deal of time tracking down the root cause of software errors -- both on and off the job. Much of the investigation effort could have been avoided if I were not presented with incomplete or incorrect error messages. The text of error messages appears to be commonly overlooked by software developers and testers.One of my early test automation development tasks was to fix the error messages in a test tool. The tool was used to validate the structure of data exchanged between computer systems. In nearly all cases, this tool displayed an error dialog window stating that the data was not valid. This tool gave the user (a tester) no information about what was wrong with the data. The lack of detail in these messages required that the user manually examine the data -- bit by bit -- with a protocol analyzer. This turned investigation of errors reported by an automation tool into a tedious manual task. The tool encountered a problem but did not report the source of the


Model-Based Test Engine Benefit #4: Generate and execute new tests – and find new bugs
2007-06-08 00:49:00
The last -- and perhaps the best -- major benefit of implementing a Model-Based Test Engine (MBTE) is automation that is capable of generating and executing tests that have not previously been executed manually. Traditional regression test automation simply retraces test steps that have already been performed manually. This may find new bugs that show up in a previously tested path though an application but will not find bugs off the beaten path. In his book "Software Testing Techniques", Boris Beizer compares the eradication of software bugs to the extermination of insects. Every method you use to prevent or find bugs leaves a residue of subtler bugs against which those methods are ineffectual.- Boris BeizerSoftware Testing TechniquesWhen we apply any method to finding bugs, we will find bugs that are found by that method. However, other bugs will remain. Finding new bugs requires variation in testing, not repeating the same thing over and over. Repeatability is often advertised


Imaginary Testing
2007-10-02 19:39:00
Ever thought of a test that you haven't executed? Ever wonder if it might be valuable to add a use case, test case, or test charter? Ever feel like you spend more time talking about how to solve problems than it would take to try some of the proposed solutions?I've been involved in discussions about software bugs that take more time than it would take to fix and retest. I understand that it is important to consider the risks associated with a bug and any proposed solutions. However, some times we just need to do it.I've been in test planning meetings that take more time than it would take to execute the proposed tests. I've also wasted time performing unnecessary tests. The problem is that the usefulness of a test is usually not known until we have the results from that test.When it makes sense, stop hypothesizing and start testing.Imaginary testing is unreliable. Ben Simo QualityFrog.com Questioning Software℠
Read more: Testing

Multiple Choice
2007-09-27 20:37:00
I dislike multiple choice tests. The distractors (the wrong answers) tend to be either so wrong that it is easy to find the correct answer or so close to the correct answer that they confuse test takers that know the material. Multiple choice tests make computer scoring of students possible but they are not capable of measuring students' understanding as well as short answer or essay tests. Tests that better measure learning are harder to grade. Tests that better measure learning require sapient human judgment -- not computer-scorable multiple choice tests.Scripted software tests based on objective pass/fail criteria overly simplify testing. These tests may be easy to execute using automation or cheap labor. These tests may be easy to score. However. they are not able to provide the same value as sapient exploration of the system under test.If only we could give software multiple-choice tests to measure quality ...Testing ain't easy. Ben Simo QualityFrog.com Questioning Softwa
Read more: Choice

Better Tools for Individuals through Collaboration
2007-10-14 17:00:00
Individuals andinteractionsoverprocesses and tools- from the Agile ManifestoI spent the second half of last week at the Agile Alliance Functional Testing Tools Visioning Workshop. (How's that for a long name?) Before the workshop, I was thinking that it seemed a little oxymoronic to have an agile workshop with a focus on tools. Perhaps my thinking was triggered by my concerns about those who seem to value "agile" processes and tools (often ones they sell) more than people.Agile people are supposed to care about people and not care about tools. Right? Wrong.while there is value in the items on the right, we value the items on the left more- from the Agile ManifestoSoftware is developed by people for people. Agility involves building better software by adapting to the needs of people instead of letting processes and tools lead the way. Process and tools do best when they have a supportive role in software development. Better tools can support agiity but they cannot make anyone
Read more: Individuals , Collaboration

Are you smarter than a 3rd grader?
2007-10-05 11:37:00
"I guess you could say I like to figure out how stuff works, I just like new adventures."- Carson Page, 8 year old junior beta testerRodolfo GonzalezAMERICAN-STATESMANGood testers can be hard to find. It looks like Actel Corp has found a good one. He is young. He is smart. He has excellent growth potential. And he works cheap -- for now.Check out these stories:Austin American-Statesman: Boy, 8, evaluates software for Silicon Valley companyEETimes: 8-year-old whiz designs with FPGAsI suspect that this kid does not know many testing buzzwords. I suspect he doesn't know much about testing tools and processes. However, Carson knows how to ask "why?" and communicate with engineers."We would ask what he liked and didn't like about it and he could explain it on a very high-end level."- Mark Nagel, Actel Corp, Field Applications Engineer A tester that can think, ask questions, and communicate can go far. Ben Simo QualityFrog.com Questioning Software℠


Green Means Go?
2007-10-17 19:28:00
Traffic control devices are used on roads to help regulate the flow of traffic. When I taught defensive driving classes, I ensured that each class included a discussion about these devices. It is imperative that all drivers understand what each device means. My first Driver License test (in Germany) required that I properly identify 94 of 100 different signs to pass. There was a time that each local governing authority created its own traffic control devices. However, it was not long after the automobile became common that governments began working together to standardize these safety-critical devices. While there is no universal standard that is really followed (the USA being one of the countries that differs from most), standardization within each country (and some continents) has led to safer streets and highways.These devices include signs, signals, pavement markings, barricades, and policemen. These devices are tools used to control the flow of traffic. However, these devices cann
Read more: Green , Means

Problems: So What's On All Those Sticky Notes?
2007-10-17 00:59:00
In my previous post about the Agile Alliance Functional Testing Tools Workshop , I wrote the following:After reviewing existing tools used by agile teams: we identified software testing issues that have been solved (yellow), those that have been partially solved (orange), and those that have not been solved (pink). As I recollect, most of the solved issues were technical problems and most of the unsolved problems were people problems. Many of the partially solved problems were those for which I believe we have technical solutions but have not yet been integrated and presented in ways that best support people.In case you are wondering what problems we wrote down on these notes, Frank Maurer kindly transcribed them for the workshop participants and I have posted them below.Looking at this list reminds me of the traffic safety problem lists I made in the defensive driving classes I used to teach. As unsafe driving practices were brought up by students, I would add them to a list on the w
Read more: Notes

Finally, a tester certification test that I might like!
2007-11-17 12:45:00
I am not a fan of any of the current software tester certification programs. Perhaps it is because I take the word certification too literally -- which means that I expect it to have real meaining. When I think of certification, I usually think in lines with the IEEE's definition of certification.certificationThe process of confirming that a system or component complies with its specified requirements and is acceptable for operational use.Perhaps my thinking is biased by my past IV&V testing work. If one can go to a weekend class and become certified, then I question the value of that certification. I believe that certifications based on ability to memorize terms and practices free of context are of little value -- and may do more harm than good. Yet, the purpose of this blog post is not to provide arguments against the current crop of tester certification options. If you'd like to see some concerns about certification, read the following.Against Certification, James BachAS
Read more: Finally

The Bananananananana Principle
2007-11-24 23:11:00
... as the little boy said, "Today we learned how to spell 'banana', but we didn't learn when to stop." ... In honor of that little boy, we can elevate his idea to a principle, The Banana Principle : Heuristic devices don't tell you when to stop. - Gerald M. Weinberg, An Introduction to General Systems ThinkingI just had the following exchange with my 12 year old daughter Jessica.Me: How do software testers know when to stop testing something?Jessica: When you die! . . . Or when you get really tired of it.[Update: The Banana Principle does not mean that heuristics cannot be useful in determining when to stop. It means that heuristics do not tell us when to stop using the heuristic. There is a tendency to start transforming the most useful heuristics into laws -- in our minds. Heuristics should help us think and not replace thinking. This includes continual questioning of even the most useful heuristics.] Ben Simo QuestioningSoftware.com


Maybe it would be better if code changes broke our tests
2007-11-23 17:14:00
For those that put faith in code coverage metrics:Consider this: If a small change made to the code produces no change in the results of any of the tests , we have evidence of insufficiency of the full set [of tests].- Robert H. Dunn,Software Quality: Concepts and Plans Ben Simo QuestioningSoftware.com
Read more: Maybe , better , changes , broke

Arranging Abstract Absolute Artifacts
2007-11-22 11:30:00
For any system of interesting size it is impossible to test all the different logic paths and all the different input data combinations. Of the infinite number of choices, each one of which is worth of some level of testing, testers can only choose a very small subset because of resource constraints.- Lee Copeland,A Practitioner's Guide to Software Test DesignThe complexity of software makes it impossible to test all the possible things we could test for all but the most simple systems. (And I have often argued that even very simple systems cannot be completely tested.) This inability to test everything requires that we testers (and testing developers) identify the things that we believe are most likely to help us fulfill our testing mission. This makes test design very important. We not only need to design our tests in a way that supports our mission -- we need to communicate our testing in a way that supports our mission.There are many ways that we can design and document tests
Read more: Abstract , Absolute

What is Software Testing?
2007-11-27 18:52:00
I know what I mean when I say testing, but what do you mean when you say testing?Bill Cosby does a comedy routine on his 1964 album "I Started Out As A Child" about a patient in surgery (under local anesthesia) hearing the doctor say "Oops!" The patient says "What did you say?! What did you say?! Did you say oops?! I know what I've done when I say oops! What did you do saying oops there?!"Hopefully none of us ever hears our doctor say "Oops!" while we are being treated. As ambiguous as "Oops" is, I suspect that any English speaker hearing it understands that it indicates that the speaker has accidentally done something bad. However, some other words with seemingly less ambiguous definitions can lead to misunderstandings and conflict amongst people.This confusion can be caused by both the natural ambiguity of language and each person's experience and understanding. I believe the words "test" and "testing" are great examples of words that should cause us to ask what definitions are bei
Read more: Software , Testing , Software Testing

Solving Intractable Problems
2007-12-04 21:10:00
"A solution to a given problem is called optimal if one can prove that no better solution exists. Some skeptics might ask, Why should intuition rely on a rule of thumb instead of the optimal strategy? To solve a problem by optimization -- rather than by a rule of thumb -- implies both that an optimal solution exists and that a strategy exists to find it. Computers would seem to be the ideal tool for finding the best solution to a problem. Yet paradoxically, the advent of high-speed computers has opened our eyes to the fact that the best strategy often cannot be found." - Gerd Gigerenzer, Gut Feelings: The Intelligence of the UnconsciousAn intractable problem is a problem for which there is no efficient means of solving. These aren't necessarily problems for which there is no solution. Instead, these are problems that take too long to analyze all the options. One of the biggest challenges in testing software is to select useful tests from infinite options. Even finite sets of options


Whose side are you on?
2007-12-16 16:35:00
Many marriages would be better if the husband and the wife clearly understood that they are on the same side.- Zig ZiglarMany software teams would be better if the testers and developers clearly understood that they are on the same side.No one creates a team to battle their team members. Husbands and wives don't embark on life together so that they can spend their lives battling each other. Software development teams aren't formed to create conflict within a company. Yet, conflict happens. Partners become the victims of friendly fire.When the going gets tough, the tough need to stop bickering long enough to realize that they are on the same team. Here are a few tips that can help keep a team together.Keep the common mission in sightSeek first to understand, then to be understood (Stephen Covey)Help others overcome obstacles instead of assigning blamePraise good workBe honest and tactfulAlthough each team member may serve a different role, it is essential that teams value the inpu


Discombobulation Testing
2007-12-22 01:31:00
I do not like software that discombobulates users. I find that software error handling and reporting often confuses users more than it helps. Several months back, I created and wrote about a mnemonic of guideword heuristics that I have been using for testing error handling and reporting. I have found this useful in my own testing and have received some positive feedback from others that are using it. I created the following PowerPoint show to help demonstrate how I use the mnemonic. Take a look. It runs for just under seven minutes.FAILURE.pps(For those that don't have PowerPoint: try an experimental Flash version here.)The next time you encounter an error, try the FAILURE heuristic.F Does the error detecting, reporting, and handling function as expected?A Is the error reported at the appropriate time in an appropriate manner?I Is the impact to the user communicated?L Is the error appropriately logged? ... or not logged if that is desired?U Is the error appropriately reported i
Read more: Testing

Page 4 of 5 « < 2 3 4 5 > »
Copyright 2006 - 2007. TopBlogArea.com All Rights reserved. Created by SEOMinds.com
eXTReMe Tracker