是否有任何类似的方法可以在 pyspark 中复制 pandas 的“qcut”function？

Question

I wanted to do the KS test in the pyspark for the predicted probability and true labels. The similar work has been done in the pandas in the link: https://www.listendata.com/2019/07/KS-Statistics-Python.html

Answer 1

No there is no direct way. You have to apply window functions etc. I have always convert to pandas when I needed this:-) Or when I am working in a Databricks type environment I leverage spark sql. I have found these easier than the windowing methods.