如何为PySpark ETL设置本地开发环境以在AWS Glue中运行？

Question

PyCharm professional supports connecting, deploying and remote debugging of AWS Glue developer endpoint ( https://docs.aws.amazon.com/glue/latest/dg/dev-endpoint-tutorial-pycharm.html ) , but I can't figure out how to use VS Code (my code editor of choice) for this purpose. Does VS Code support any of these functionalities? Or is there another free alternative to PyCharm professional with the same capabilities?

Answer 1

I have not use pyCharm, but have setup a local Development End Point with Zeppelin, for my Glue jobs development / testing. Please see my related posts & references for setting up local development end point. Maybe you can try it, if it is useful, and you can try to use pyCharm instead of Zeppelin.

Reference : Is it possible to use Jupyter Notebook for AWS Glue instead of Zeppelin & Link for zeppelin local development endpoint SO discussions