Azure Synapse development (code writing) best practice

Question

our group is starting to work on Azure Synapse for our data analytics and we've seen the integrated notebooks and sql. however, these are very minimal and might be good place to start, but soon enough we'll want to move the work to some IDE like VSCode or PyCharm...

any best practices on how or which IDEs are integrating with my Azure Synapse code? any of them where i can run sql or notebooks from my local IDE to the synapse account?

and if on the subject any best practices on developing in Synapse at all regarding how to write and test my work before firing it in the environment?

Answer 1

First of all install spark and hive tools in visual studio code. Then we can walk through how to submit jobs to Spark & Hive tools.

Spark and hive tools can be installed on platforms which are supported by visual studio.

Following items are required to complete the steps involved in moving notebooks from local IDE to the synapse account.

Serverless apache spark pool

Visual Studio Code

A Pyspark interactive environment for Visual Studio Code

Steps involved in order to process or move sql / notebooks from local IDE to the synapse account.

So here after meeting the above prerequisites you can install Spark & Hive Tools for Visual Studio Code by following these steps:

1. Open Visual Studio Code.
2. From the menu bar, navigate to View > Extensions.
3. In the search box, enter Spark & Hive.
4. Select Spark & Hive Tools from the search results and then select Install. 5.Select Reload when necessary.

please find the below link for more details.

https://docs.microsoft.com/en-us/azure/synapse-analytics/spark/vscode-tool-synapse