Testing Procedures? Testing at all?

As a Test Director on a number of defense contractor hardware and software projects I have to ask:

  • What test procedures were written?
  • What audience was used for alpha testing?
  • What audience was used for beta testing?

This outcome is really pretty poor. It’s cumbersome to learn, it’s cumbersome to set up, it’s cumbersome to change the (trial) setup. It’s cumbersome to set a browser shortcut to get to back to (only) the selected boards–because it’s too hard to identify the Web page that needs to be included in the browser’s shortcut. Which aren’t even boards anymore; you’ve buried those deeper in your menuing facility.

Why, for instance, can’t we have the table of “boards,” an ADD button, and the table of selected “boards” that we had under the old, perfectly functional way? What problem with that sort of procedure is this new, more complex/cumbersome procedure intended to fix?

Even in creating this Help/Feedback post, the only “optional tags” available for selection is “my-dumbest-investment x1.” And “Your topic is similar to…” Testing lows and Volvo Trucks Moves into H2 Testing? When the basic topic is Help/Feedback? How does that work, exactly? Your algorithm needs to be much more discriminating. Again, I ask about your testing process.

It’s true enough that for a test director every software problem looks like a testing failure, but still…

Eric Hines

7 Likes

TMF didn’t write the software, they bought a package and then worked to configure and adapt it to their needs. All packaged software is a compromise between what you want and what they offer*. When you buy packaged software the worst things you can do is try to do things YOUR way instead of the way the software was designed to work.

I’m finding our new home (actually homes, as I am also on the paid Premium Community side) confusing, and I’m still wandering around lost some of the time. But I’ve also managed to reach the point where I can read every new message for all of what used to be my Favorite Boards, missing nothing. I’m sure the finer points that I need - a much smaller subset than what is available - will work themselves out for me.

As to their testing… they moved the Options premium (paid) service first, months ago. I’m not on that, and I guess they learned some things and missed some others. It appears to me that most of the issues in getting around related to there being so many more places to choose from than there would be with just Options. And no amount of testing would make people happy when what they want isn’t available in the package being implemented.

*(Just as all software is a compromise between conflicting requirements and preferences.)

3 Likes

Absolutely correct @RHinCT - we tried to strike a balance between what we knew was important to our users and and not painting ourselves into a corner by over-customizing.

1 Like

And

Both absolutely true. Which raises the question: if the software doesn’t do what the buyer wants it to do, why buy it? If the seller can’t, or refuses to, modify his software to comport with the buyer’s (carefully developed) explicit requirements, why do business with that seller?

Buyer testing before rollout would have revealed the vast majority of the problems the user community now is discovering/exposing in real time. With that testing outcome, the buyer could have gone back to the seller for modifications–or at the very least developed much better documentation for the user community before rollout.

Eric Hines

2 Likes

Some problems require software changes. Some require changes to the configuration of a software package. Some require guidance and training of the users.

In the end we need to look forward and learn how to use what we have.

3 Likes

Hi Eric,

If you go into preferences, you just scroll through the list of categories highlighting the ones you want to add, its pretty quick. Basically as easy as a check box.

If you decide to add a new favorite while reading a category, you have a bell (instead of a heart) at the top of the page. And, the bell allows you to mute aka unfavorite a category (actually 5 levels of favoriting really) that the old system didn’t offer.

Vicki

1 Like

What is important to your users? The new website added features (search, images, edit, notifications), but did not keep some important features (quickly finding useful content). No one reads every post, and so the way a forum does filtering is key.