31 Days of Mango | Day #13: Marketplace Test Kit

This article is Day #13 in a series called 31 Days of Mango, and was written by guest author Dave Bost. Dave can be reached on Twitter at @davebost.

---

The final journey for your Windows Phone application development is to publish your app in the Marketplace to share with Windows Phone users across the globe. The Marketplace submission process is pretty straight forward, however there are requirements that each and every app must pass before being certified in the Marketplace. The process for certifying your app could take days if not weeks depending on how many requirements your app may fail during certification testing. The time to certification can be cut down substantially with the new Marketplace Test Kit included in the Windows Phone SDK 7.1.

Note: Every Windows Phone developer needs to understand the Application Submission Requirements.

Running your Windows Phone project through the Marketplace Test Kit helps you determine if your application is ready for submission into the Windows Phone Marketplace. Prior to the availability of the Marketplace Test Kit, developers would upload their app to the Marketplace and they would have to wait for a period of time before they found out if their app passed certification or not. Sometimes these submission errors were as simple as submitting the debug version of your application package or a little more complicated such as not handling Tombstoning properly. With the Marketplace Test Kit, the developer is able to run through a battery of tests to determine if their application is ready for certification prior to going through the Marketplace submission process. This saves the developer a tremendous amount of time and gets your application in the Marketplace faster.

The Marketplace Test Kit is included in the Windows Phone SDK 7.1. You can access the Marketplace Test Kit by opening your application solution in Visual Studio, highlighting the project you’d like to test and opening the Marketplace Test Kit from the Project menu.

The Marketplace Test Kit is kept updated by the changing certification requirements of the Windows Phone Marketplace. If an update is available, you will be prompted to apply the update when you open the Marketplace Test Kit. The test kit is applied on a per Project basis. If you have multiple Windows Phone application projects in your solution, you will need to run the test kit for each Project.

Once the Marketplace Test Kit is open, you will see a series of tabs that define application details and multiple test categories. These include: Application Details, Automated Tests, Monitored Tests, and Manual Tests.

The Application Details tab contains details such as the path to your application package (location of your .XAP file) as well as images that represent your application in the Marketplace (Marketplace tile and Screenshots) and on the phone (Large application tile and Small application tile). Note that the path to your application package must be the path to the Release build of your project. There is an MSDN How-To article that outlines the process for creating a proper Release build for your Windows Phone project.

One of the tests in the Automated Tests tab is to verify the required image sizes for your tile images and screenshots. A trivial way to create the appropriately sized screen shots is to use the new screenshot tool found within the Windows Phone SDK 7.1 Emulator.

The Automated Tests tab evaluates application size requirements, capabilities required by your application, and the iconography and screenshot requirements. Press the ‘Run Tests’ button to run the tests.

Once the automated tests are complete, you will see whether or not your tests passed or failed in the Result column. You may also receive additional information regarding the test to help resolve any failures or helpful information such as the capabilities used within your application. You can use the results of the Capability Validation test to define the necessary capabilities required by your application in the WMAppManifest.xml file.

The Monitored Tests tab lists a series of tests to analyze the performance and reliability of your application. When you press the ‘Start Application’ button, your application is deployed to your target device. From there you are to run through your apps functionality while the Monitored Tests analyze the various test cases.

When you are finished running through the functionality, you can press the ‘Close Application’ button. The Result Details column lists various messages based on the test outcome. It is advisable to set the target device to Windows Phone Device and not the Windows Phone Emulator. Performance on the emulator does not represent the actual performance of a physical Windows Phone device. Running the tests on a device will provide the most accurate results for these tests.

The Manual Tests tab list a number of test cases that require you to manually run through on your application. Once again, these tests should be performed on an actual device and not the emulator. These test cases help verify the various scenarios that can’t be tested with automated tools or scripts. These include tests such as:

• Application Responsiveness during and after various conditions
• Content and Theme changes – is your app legible in both dark and light themes?
• Toast Notification Opt-In
• Volume Control Commands for apps that use Audio
• Proper handling of Background Transfers

For each test, follow the steps outlined in the Test Description column. Based on your findings, set the Result to either Passed or Failed. Your application may not implement features required by each test condition. For these particular tests you can set the Result to Pass to maintain consistency across your test cases. However, be sure that your app doesn’t implement any of the features of those passed over test cases. You don’t want to risk failing certification because of a shortcut taken in your testing procedures.

Once your Project passes all of the tests in the Marketplace Test Kit, you can be assured that your app is primed and ready to accelerate through the certification process.

Summary

The Marketplace Test Kit saves the developer substantial time in getting their app published. With the thoroughness of the automated, monitored and manual tests, Windows Phone developers can expect an expedient process to getting their apps in the Marketplace and serving Windows Phone customers from across the world.

Tomorrow, we are going to dive into OData, and how we can consume OData feeds in our applications.  It is written by guest author Chris Woodruff.  See you then!