Spark Scala-將具有一條記錄和一列的Dataframe轉換為Double

Question

我寫的scala代碼給了我數據類型錯誤。 testpredict_02的主要方法采用Double。

val featuresMD = hiveContext.read.parquet("hdfs://machine01:9000/models/nb/metadata/features")

def testpredict_02(VData: Vector) = { MyModel.predict(VData) }

def outerpredict_02(argincome: String,argage: String,arggender: String) = { 
featuresMD.registerTempTable("features_md")

val income = hiveContext.sql("select distinct income_index from features_md where income = argincome")
val age     = hiveContext.sql("select distinct age_index from features_md where age = argage") 
val gender  = hiveContext.sql("select distinct gender_index from features_md where gender = arggender") 

testpredict_02(Vectors.dense(income.select("income_index"), age.select("age_index"), gender.select("gender_index")))

---

Error :
<console>:43: error: type mismatch;
 found   : org.apache.spark.sql.DataFrame
 required: Double
              testpredict_02(Vectors.dense(income.select("income_index"), age.select("age_index")))

請幫忙..

Answer 1

如果確定3個數據框中的每一個都只包含一個列和一個記錄，則可以為每個數據框獲取第一條記錄的第一列：

def getFirstCell(df: DataFrame): Double = df.first().getAs[Double](0)

val vector: Vector = Vectors.dense(
  getFirstCell(income.select("income_index")),
  getFirstCell(age.select("age_index")),
  getFirstCell(gender.select("gender_index"))
)

testpredict_02(vector)