用于Scala和Spark的多类分类的自定义损失函数

Question

I want to ask is this possible to write a custom loss function for Multi class Classification in Spark using Scala. I want to code multi-class logarithmic loss in Scala. I searched Spark documentation but could not get any hint.

Answer 1

From the Spark 2.2.0 MLlib guide :

Currently, only binary classification is supported.. This will likely change when multiclass classification is supported.

Answer 2

If you are not restricted to a particular classification technique I would suggest using XGBoost. It has a Spark-compatible implementation, and it makes it possible to use any loss function provided you can compute is derivative twice.

You can find a tutorial here .

Also the explanation about why it is possible to use a custom loss function can be found here .