convert breeze.linalg.DenseMatrix[Double] to a dataframe in Scala

Question

i have a breeze.linalg.DenseMatrix[Double] as follow, i want to convert it to a dataframe

breeze.linalg.DenseMatrix[Double] =
0.009748169568491553   3.04248345416453E-4    -0.0018493112842201912  8.200326863261204E-4
3.0424834541645305E-4  0.00873118653317929    6.352723194418622E-4    1.84118791655692E-5
-0.001849311284220191  6.35272319441862E-4    0.008553284420541575    -6.407982513791382E-4
8.200326863261203E-4   1.8411879165568983E-5  -6.407982513791378E-4   0.008413484758510377

is there any way i can do that?

Answer 1

after couple times of try, i am able to create a dataframe that contains the flattened information of the matrix. and create a tempview, so that access from python as a dataframe

in scala

// covarianceMatrix (in scala)

val c = covarianceMatrix.toArray.toSeq
val covarianceMatrix_df = c.toDF("number")
covarianceMatrix_df.createOrReplaceTempView("covarianceMatrix_df")

in python

covarianceMatrix_df=spark.sql('''SELECT * FROM covarianceMatrix_df  ''')
covarianceMatrix_pd = covarianceMatrix_df.toPandas()
nrows = np.sqrt(len(covarianceMatrix_pd))
covarianceMatrix_pd = covarianceMatrix_pd.to_numpy().reshape((int(nrows),int(nrows)))
covarianceMatrix_pd