Gluon Mobile VideoService doesn't work on iphone in background

Question

I'm using VideoService to playback a local audio file (mp3) in my application, and it works fine while the application is active, both on Android and iPhone. But on iPhone, when the application is in background it doesn't work: nothing happens when service.play() is called. The code is trivial:

            Services.get(VideoService.class).ifPresent(service -> {
                service.getPlaylist().add("1.mp3");
                service.play();
            });

I can see "AVPlayer hidden" and "AVPlayerStatusReadyToPlay" in my IDEA's console.

If the playback is already started and I put my application in background (using iPhone's "home" button or by turning off the screen) - it stops playing and resumes only after I bring the application back to active state manually.

JavaDocs say no specific iOS configuration is required, though I put "audio" in plist's UIBackgroundModes array (doesn't help either).

iPhone 6, iOS 12.1 (16B92)

On Android the same code works just fine both in active and background modes without any problems.

What am I missing?

Answer 1

To be able to play audio in background, and based on this answer , it seems that the current Charm Down Video Service requires some modifications, in order to set the category to AVAudioSessionCategory.Playback .

One possible way to do this, is by modifying applicationDidFinishLaunching , from the iOS Launcher class. The jfxmobile plugin 1.3.16 creates this launcher here .

So we could modify that class, and build a custom version of the jfxmobile plugin, or, as the OP mentioned in the comments, it is possible to create a custom launcher.

The other possible way is adding this to the Charm Down video service directly, but it will require compiling a new version.

Let's try the custom launcher as it won't require building new versions.

Creating a CustomLauncher

Let's copy the default launcher into our project, into the src/ios/java folder, as it requires some specific dependencies for iOS.

Then add the required code to set the Playback option, starting from the main class:

private static final Class<? extends Application> mainClass = your.package.YourGluonApplication.class;

private static final Class<? extends Preloader> preloaderClass = null;

@Override
public boolean didFinishLaunching(UIApplication application, 
    UIApplicationLaunchOptions launchOptions) {

    // Audio settings to play in background mode ---
    try {
        AVAudioSession session = AVAudioSession.getSharedInstance();
        session.setActive(true);
        session.setCategory(AVAudioSessionCategory.Playback);
    } catch (NSErrorException nse) {
        System.out.println("Error AVAudioSession " + nse);
        nse.printStackTrace();
    }
    // --- End Audio settings

    Thread launchThread = new Thread() { ... }
    launchThread.setDaemon(true);
    launchThread.start();

    return true;
}

Using the custom launcher

As commented in the launcher class, the custom launcher can be loaded from the build.gradle file:

jfxmobile {
    downConfig {
        version = '3.8.6'
        plugins 'display', 'lifecycle', 'statusbar', 'storage', 'video'
    }
    ios {
        javafxportsVersion = '8.60.11'
        launcherClassName = 'your.package.CustomLauncher'
        infoPList = file('src/ios/Default-Info.plist')
        ...
    }
}

Allow background audio

The last required step to get audio playing not only when the app is running on the foreground but also when it goes to the background: modify the plist file.

We need to add this key to the Default-info.plist file:

<key>UIBackgroundModes</key>
        <array>
            <string>audio</string>
        </array>

Test

Let's add an mp3 file to src/main/resources/ , like 1.mp3 , and include this call in our Java code:

Services.get(VideoService.class).ifPresent(service -> {
            service.getPlaylist().add("1.mp3");
            service.play();
        });

Time to deploy to an iOS device:

./gradlew launchIOSDevice

The app should play the audio in both foreground and background mode as expected.