Device Test Matrix
A structured test grid for every major OTT platform and critical scenario.
How to Use This Matrix
Copy this matrix into a Google Sheet or Notion table. For each device, mark each scenario as Pass, Fail, or N/A, and add notes for any failures. Attach the completed matrix to your go-live checklist before submitting to app stores.
Core Test Scenarios
| # | Scenario | What to check |
|---|---|---|
| 1 | Launch / cold start | App opens in < 3 seconds |
| 2 | Sign in (email/password) | Login completes, session stored |
| 3 | Sign in (second screen) | QR code / URL flow works end-to-end |
| 4 | Homepage load | Featured content and rails display correctly |
| 5 | Catalog browse | Category navigation works via remote |
| 6 | Search | Results appear; correct titles returned |
| 7 | Detail page | Title info, trailer, and play button visible |
| 8 | Playback start | Video plays within 3 seconds of pressing play |
| 9 | Seek | Forward / backward seek works via remote |
| 10 | Resume playback | Returns to last position after pause / re-open |
| 11 | Quality adaptation | Buffers gracefully at simulated low bandwidth |
| 12 | Subtitles / CC | Toggle and render correctly |
| 13 | Purchase flow | Subscription initiates and completes |
| 14 | Entitlement check | Non-subscribers are blocked before paywall |
| 15 | Remote: back button | Returns to previous screen; does not exit app |
| 16 | Remote: home button | Returns to TV home without crashing |
| 17 | App backgrounded | Session preserved when returning to app |
| 18 | Network interruption | App reconnects and resumes gracefully |
| 19 | 60-minute session | No crash, freeze, or memory degradation |
| 20 | Sign out | Session cleared; login screen shown |
Device Coverage Grid
| Scenario | Web | iOS | Android | Tizen | webOS | Roku | Fire TV | Android TV | Apple TV |
|---|---|---|---|---|---|---|---|---|---|
| Cold start | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Sign in | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Second screen sign in | — | — | — | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Browse | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Playback | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| DRM | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Purchase | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Back button | — | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Network recovery | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| 60-min session | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
(Blank cells = test not applicable for that platform)
Severity Classification
Use these severity levels when logging failures:
| Level | Definition | Action |
|---|---|---|
| P1 — Blocker | App crashes, payment fails, DRM fails | Block submission; fix immediately |
| P2 — Major | Key flow broken (sign in, playback, purchase) | Fix before submission |
| P3 — Minor | UI misalignment, cosmetic issue | Fix before launch; note in submission |
| P4 — Nice-to-fix | Edge case, cosmetic only | Log for post-launch sprint |
Tip: Maintain one row per physical device (not just platform). Two Samsung TVs from different years can behave very differently. Test on the oldest supported hardware you intend to certify.
Want a formatted version of this matrix?
Book a call and we will share a Google Sheets version you can adapt for your specific device list.