Tag Archives: QA

20 Questions Product Managers Should Ask about Mobile App Testing

I’ve worked as a product manager on several different types of mobile apps: Android, HTML5, iOS, phone, tablet…but always B2B. Building a new mobile app is a great experience. During testing, changes can be made very quickly to overcome weaknesses of the app.

The most challenging part of building mobile apps is building to the devices. Devices have limitations. Not everyone buys a new, high-priced phone or tablet every two years. Therefore, both the test plan and the project plan for a mobile app are dramatically different from a browser-based version of the same workflow. Product managers need to know the different scenarios in which the user runs the app, and they need to convey that information to the team. Below are some good questions to ask the team:

1. What are some ways we are testing security?

2. How are we testing with different screen sizes?

3. How are we testing the install and uninstall events?

4. Which carriers are we testing with?

5. Are we testing with no storage space available on the device?

6. Are we testing with low battery?

7. Are we testing with “dirty” devices (OS not upgraded, apps not upgraded, generally neglected)?

8. Are we testing with low-priced devices?


9. Are we testing with incoming calls?

10. Are we testing with the devices in different positions, alternating between portrait and landscape?

11. Are we testing with screen covers?

12. Are we testing with different pixel densities?

13. Are we testing using public Wi-Fi, maybe at Starbucks?

14. Are we testing with different screen brightness settings?

15. Are we testing whether we can walk and use the app?

16. Are we testing whether we can use just a thumb to use the app?


17. Are we testing with headphones plugged in to the device?

18. Have we looked at https://appthwack.com for automated testing?

19. Have we looked at http://www.perfectomobile.com for automated testing?

20. Why will people not like using this app?

 

I hope you enjoyed the post. Leave a comment on other questions to ask. For more activities for product management, read the Product Guy Daily. I publish it every morning.

New Product Managers: How a Simple UX Project is a Great Way to Get Started

Congratulations; you are a new product manager! Product managers need to pick the brains of existing users and potential users of products. They need to understand the different personas of users. They need to understand pain points of using the product and looking at the product. They need to have users that they can ask clarifying questions over the course of a year. If a new product manager does a simple UX project (like this one), she will have all those things early.

 

Recruiting

Recruit 5 expert users of the product. Recruit 5 users familiar with the product. Recruit 5 potential users of the product.

 

Questions

Have the participants answer a few questions about themselves and then have them complete about 10 common tasks using the product. Ask their name, age, and any other relevant demographical data. Ask their experience level: (a) I’ve never used it and never seen it, (b) I’ve never used it, but I’ve seen it, (c) I’ve used it once or twice, or (d) I’ve used it weekly or more often. Have them briefly take a look at the product. What do they think it is there to do?

 

Tasks

Start using a screen capture tool. I’ve used Screenr and CamStudio, and both are acceptable. Screenr is better. If data security is a concern, be careful with what you’re recording and where you’re storing it. Ask them to perform ten tasks. This should not take them more than 30 minutes.

 

More questions

Ask them, on a scale of one to ten, how easy it is to use a product. Ask them how the product could be improved.

 

Conclusions

The most difficult parts of the project will be fine-tuning the script and recruiting the users. Both of these activities are very valuable to a new product manager. By watching the recordings, figure out how long it took them to perform tasks. Look for patterns in the problems the participants experienced. Look at the quotes from participants and use them. Lastly, write a short paper with recommendations on UX improvements and translate the recommendations into business requirements and functional stories.

 

For more information on product management and UX, read the Product Guy Daily. I publish it every morning.

10 Consequences of Having Product Managers as Primary Testers

Sometimes a product manager is asked to do things like testing to help software development processes. This is especially true for product management job titles like product owners, business analysts, product analysts, etc. What about when the product manager is the primary tester? Sure, there are reasons why it can be a good thing. First and foremost, it is a great way for a product manager to understand the product and how it is used. However, I wanted to focus on the limitations of this arrangement. I’ve compiled a few of the consequences that arise from having product managers as the primary testers. These include impacts on the development process, conflicts of interest, and impacts on the performance of the product manager.

 

Impacts on the Development Process

  • The turnstile. Using the product manager as a tester is to have one person be the action person for too many parts of the process. You’ll be at the beginning, the middle, and the end of the team’s work. In this flow, the product manager naturally becomes a bottleneck.
  • The weird smell. You’ll be less likely to investigate something that passes, but passes oddly (something’s not right). In an ideal world, an engineer or tester should investigate the weird smell. A product manager probably doesn’t have the expertise or the time.
  • The importance of testing. Testing is a full-time job (usability testing, performance testing, load testing, security testing, etc.). Doesn’t there need to be a dedicated person doing this job?
  • The stuntman argument. If you’re an actor, you may not want to do your own stunts because it takes away work from a stuntman. Same argument here: you’re taking away experience from a junior engineer.

 

Conflicts of Interest

  • Pulled in different directions. Product managers should want to ship the product as soon as possible; testers should want to send problems back to engineers to ensure quality. Product managers who test face tough decisions on what is important.
  • Scope creep. The product manager will have the ability to sneak in additional requirements to get it perfect. It’s always very tempting for a product manager to do.
  • Perfectionism. Being the finder of bugs makes you more likely to delay a launch to address a defect. After all, finding and squashing the bugs is what a good tester does.

 

Impacts on Your Performance as a Product Manager

  • Less time for your day job. You’ll have less time to interface with customers. Understanding customers is probably the most important thing for a product manager to do.
  • Different mindset. Testing puts you more in the details, and less with the overall vision. Testing every day can shift focus away from developing the 3-year big picture, or at least developing the big picture with an open mind.
  • One more way to be wrong. As the product manager, you’ll be the throat to choke for starting late. As the tester, you’ll be the throat to choke for the launch being delayed. Why open yourself up to more potential criticism?

 

 

What do you think about product management and testing? Leave a comment!

 

For more information on product management, read the Product Guy Daily. I publish it every morning.