Difficulty using java library in Android Studio, works fine in IDEA

Question

new Android programmer here. I've been working on an android app for the past couple days and I've run into a bit of a road block.

I'm using a library called Kuromoji ( http://www.atilika.org/ ) to parse Japanese text blocks into individual words. But it doesn't seem to be functioning correctly when I utilize it in Android Studio.

It builds and runs just fine in Intellij IDEA with pure Java, as well as a Java project with Gradle, but it starts malfunctioning when I move to Android Studio.

I'm just running newly generated empty projects for both IDEA and Android Studio, I can't seem to pin down the conflicting factor.

Thanks for reading, any help would be greatly appreciated.

import org.atilika.kuromoji.Tokenizer;
import org.atilika.kuromoji.Token;

public class kuromoji_test {

public static void main(String[] args){
    Tokenizer tokenizer = Tokenizer.builder().build();
    for (Token token : tokenizer.tokenize("寿司が食べたい。")) {
        System.out.println(token.getSurfaceForm() + "\t" + token.getAllFeatures());
    }
}
}

Running in IDEA with or without Gradle, this code produces the correct parsed words along with part of speech etc...

寿司 名詞,一般, , , , ,寿司,スシ,スシ'

が 助詞,格助詞,一般, , ,*,が,ガ,ガ

食べ 動詞,自立, , ,一段,連用形,食べる,タベ,タベ

たい 助動詞, , ,*,特殊・タイ,基本形,たい,タイ,タイ

。 記号,句点, , , , ,。,。,。

package com.example.towson.kuromoji_testing_with_android_studio;

import android.support.v7.app.ActionBarActivity;
import android.os.Bundle;
import android.view.Menu;
import android.view.MenuItem;

import org.atilika.kuromoji.Token;
import org.atilika.kuromoji.Tokenizer;


public class MainActivity extends ActionBarActivity {

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    //setContentView(R.layout.activity_main);

    Tokenizer tokenizer = Tokenizer.builder().build();
    for (Token token : tokenizer.tokenize("寿司が食べたい。")) {
        System.out.println(token.getSurfaceForm() + "\t" + token.getAllFeatures());
    }

}


@Override
public boolean onCreateOptionsMenu(Menu menu) {
    // Inflate the menu; this adds items to the action bar if it is present.
    getMenuInflater().inflate(R.menu.menu_main, menu);
    return true;
}

@Override
public boolean onOptionsItemSelected(MenuItem item) {
    // Handle action bar item clicks here. The action bar will
    // automatically handle clicks on the Home/Up button, so long
    // as you specify a parent activity in AndroidManifest.xml.
    int id = item.getItemId();

    //noinspection SimplifiableIfStatement
    if (id == R.id.action_settings) {
        return true;
    }

    return super.onOptionsItemSelected(item);
}
}

Everything builds just fine, but now Kuromoji simply returns my input by character. The other function that returns dictionary info doesn't seem to be running at all.

Kuromoji is in my libs folder, and I added the jar as a library.

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ 寿

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ 司

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ が

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ 食

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ べ

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ た

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ い

05-10 01:06:52.813 5758-5758/com.example.towson.kuromoji_testing_with_android_studio I/System.out﹕ 。

my build.Gradle from Android Studio :

apply plugin: 'com.android.application'

android {
compileSdkVersion 22
buildToolsVersion "21.1.2"

defaultConfig {
    applicationId "com.example.towson.kuromoji_testing_with_android_studio"
    minSdkVersion 19
    targetSdkVersion 22
    versionCode 1
    versionName "1.0"
}
buildTypes {
    release {
        minifyEnabled false
        proguardFiles getDefaultProguardFile('proguard-android.txt'),    'proguard-rules.pro'
    }
}
}

repositories {
mavenCentral()
}

dependencies {
compile fileTree(dir: 'libs', include: ['*.jar'])
compile 'com.android.support:appcompat-v7:22.0.0'
}

kuromoji is compiled in the libs folder

my build.Gradle from Intellij

apply plugin: 'java'

sourceCompatibility = 1.5
version = '1.0'

repositories {
mavenCentral()
}

dependencies {
testCompile group: 'junit', name: 'junit', version: '4.11'
}

EDIT: Library continues to malfunction when using IDEA for an Android app. Problem seems to be related to Android and not the IDE.

Checking in debug mode, class objects from my library file are instantiating correctly. Most of the string methods just return "" though. Not sure what's going on here.

Edit 2:

Here's a photo of my token and tokenizer class objects. This is what they look like when they're running correctly.

Here's a photo of my token and tokenizer class objects running incorrectly, you can see the the "surface_form" tag is only holding one character instead of two.

Does anyone know a way I can go about debugging this?

Answer 1

Alright, just figured it out. I've been using a package that makes calls to java functions not included in the Android version of Java.

That's why I've only been having problems when my project was set-up as an Android app.

Copied this from the lint report:

*Used a different set of libraries since I posted this question. Kuromoji is a Japanese morphological analyzer and has since donated their code to the Apache Software Foundation.

Using org.apache.lucene libraries in this report.

InvalidPackage: Package not included in Android

../../libs/lucene-analyzers-common-5.1.0.jar: Invalid package reference in library; not included in Android: java.nio.file. Referenced from org.apache.lucene.analysis.hunspell.Dictionary.

../../libs/lucene-core-5.1.0.jar: Invalid package reference in library; not included in Android: java.lang.invoke. Referenced from org.apache.lucene.util.AttributeFactory.

Priority: 6 / 10

Category: Correctness

Severity: Error

Explanation: Package not included in Android. This check scans through libraries looking for calls to APIs that are not included in Android.

When you create Android projects, the classpath is set up such that you can only access classes in the API packages that are included in Android. However, if you add other projects to your libs/ folder, there is no guarantee that those .jar files were built with an Android specific classpath, and in particular, they could be accessing unsupported APIs such as java.applet.

This check scans through library jars and looks for references to API packages that are not included in Android and flags these. This is only an error if your code calls one of the library classes which wind up referencing the unsupported package.