Android Alpha/Beta testing

Question

My company will be releasing a communication app for Android in the coming weeks/months. It is an embedded app on a Nexus 7 2013 device, running AOSP with some minor modifications. This makes typical remote updates complicated as there's an extra step we must run to install the APK/libs into the OS system folders.

The plan is to do Alpha and Beta trials, and we have started gathering volunteers. We'll be loading the APK onto devices and giving them out initially. I've found plenty of information on generally preparing for release (unit testing, signing your APK, etc) but I'm looking for information on Alpha and Beta releases.

Problems:

• We need to update the app remotely without bringing the devices in
• We need to gather as much (useful) data as possible on usage, device statistics, bugs and crashes

Research & Possible Solutions:

• Developer Console Alpha/Beta testing - https://developer.android.com/distribute/googleplay/developer-console.html
• App Blade or other third party support tools - https://appblade.com
• Analytics libraries
  • Flurry - https://developer.yahoo.com/analytics
  • Mixpanel - https://mixpanel.com/android-analytics
  • Google Analytics - https://developers.google.com/analytics/devguides/collection/android/v4/
  • Amplitude - https://amplitude.com/behavioral-analytics-platform
  • Localytics ($) - https://www.localytics.com
• Fetch logcat programmatically on the device and upload to our server
• Save specific in-app logs/statistics to internal SharedPreferences or SQLite database and upload to our server periodically
• Gather written feedback from Alpha/Beta testers via surveys, email communication, etc.
• If the device is in hand, we can debug crashes by:
  1. Dump logcat to file - "adb logcat -d > logcat.txt"
  2. Check kernel logs - /proc/last_kmsg
  3. Check dump - /data/dont_panic

Questions:

• Have you run an Alpha/Beta trial using Google Play Developer Console? Was it useful?
• Have you used any third party tools such as App Blade to push releases and gather statistics?
• What are the best practices for gathering detailed statistics/feedback from your apps once they've been released?

Thanks in advance

Answer 1

1. Alpha/Beta tests are extremely useful. You can create google+ communities for Alpha and beta testers. Then add those community for sharing the app with specified users. A new version can be easily deployed and notified to alpha and beta users.
   1. Nups
   2. Multiple ways actually: Write your own beacon kind of system to gather information from device and send those using Alarm Manager periodically. For storing logs, sql should be used.

Answer 2

1. Yes. I've tried running a private Alpha and a public beta using the dev console and G+ groups. This is really useful if the testers are used to g+ and how it works as we gathered feedback there. Also this is the only way that I know of to seamlessly "push" the updates to the users. The only downside is that the dev console lacks an api so you can't upload builds automatically from a ci-server.

2. Tried Test Flight before it was bought by Apple but didn't use it for feedback, only distribution. In the most simple cases I've used Dropbox.

3. I would recommend the the following feedback "stack":

   • Crashlytics for exception reports (they also do beta distribution now but haven't tried it yet) - http://crashlytics.com
   • Google analytics for metrics like time spent on certain parts, load times etc
   • G+ groups for direct feedback