在 Azure Devops Pipeline 中运行 Python 代码并将输出导出到 Devops Repos ($(Build.SourcesDirectory)) 中的文件夹

Question

How could I rewrite this python script so that it can run within azure devops pipeline and export the dataframe as a csv to the devops repository. I'm able to achieve this locally but would like to achieve this remotely.

Put different, how can I export a pandas dataframe to devops repos folder as a csv file using an azure devops pipeline task. Below is the python script that needs to run as a pipeline task.

local_path in this case should be azure devops path.

from azureml.core import Workspace, Dataset
local_path = 'data/prepared.csv'
dataframe.to_csv(local_path)

Answer 1

⚠️You really should not do this. Azure pipelines are for building code, not for processing data. Assuming that you meant Azure DevOps Pipelines , opposed to Azure ML Pipelines .

Also you should not commit data to your repository.

If you still want to proceed, here is an example for what you try to achieve. Note that for the last line, ie git push , you need to give the agent permission to write the repository. See Run Git commands in a script for an approximate ☹️ documentation on how to do that on your account.

trigger: none

pool:
  vmImage: 'ubuntu-latest'

steps:
- checkout: self
  persistCredentials: true

- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.8'
    addToPath: true
    architecture: 'x64'

- script: |
    python your_data_generating_script.py
    git config --global user.email "you@example.com"
    git config --global user.name "Your Name"
    git add data/prepared.csv
    git commit -m'test commit'
    git push origin HEAD:master
  displayName: 'push data to master'