User has downloaded version of our app that is not released yet

Question

We just uploaded the newest version of our app to the Android Play Store Developers Console - its saved but not yet submitted for review. However, about 3 minutes after we uploaded it we got several app generated emails reporting errors from this newest version of our app. Not only do the emails identify the app version number, there is also data in the emails that can only come from this newest version because it was specifically added to this newest version - so we know 1000% the emails came from this not yet released version. The same user was able download this not yet release version was behind all error reported emails - from Nova Scotia (based on reported GPS info in error email) using a Nexus 5X on Android OS 6.0.1

How is this possible?

The app is not a part of any open or internal testing or and pre-registration or pre-launch project. And considering we haven't even submitted it for review yet this is even more troubling.

The app is on development servers is behind a very secure.network so we know it wasn't stolen (then side loaded) from our server at coincidentally the same time we uploaded it. Our PaloAlto FWs track all outbound file attachments and data streams - which shows our server only communicating with Google servers.

We have contacted Google about this - but their response time is usually between 3 and 5000 days....so who knows if we will hear anything back any time soon.

On a side note, the errors were generated because we haven't activated certain server code yet - that code is activated after the app is cleared for release.

Answer 1

Uploading an App Bundle to the Google Play Developer's Console starts automated tests running on Android devices in Google's Firebase Test Lab.

See the Pre-launch report Overview in the developer's console.

The test report gives you initial feedback on app Stability, Performance, Accessibility, and Security and Trust. This helps developers catch and fix problems before spending people time on QA.

The Performance section of the report includes the "Avg..network sent" and "Avg..network received" bytes by each test device.

Click on the right arrow for a device in that list to access the "Network sent" and "Network received" graphs in bytes/sec over the test duration. Play the recorded screen video to see what the app's UI was doing when it was sending and receiving bytes.