Scheduler makes a python script run but doesn't produce a csv file

Question

I've written a script in python to scrape some links from a webpage and write them to a csv file. My script does this in the right way when run it from an IDE.

When I run the same using windows task scheduler , I can see that a command prompt pops up and the script runs and prints result there as well but I don't get any csv file when the task is done.

Am I missing something?

What possible change should I bring about to get a csv file when the script is run via .bat file from windows task scheduler ?

.bat contains:

@echo off
"C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\python.exe" "C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\examplescript.py"

examplescript.py contains:

import csv
import requests
from urllib.parse import urljoin
from bs4 import BeautifulSoup

url = "https://www.janglo.net/component/option,com_sobi2/"

def get_info(link):
    res = requests.get(url)
    soup = BeautifulSoup(res.text,"lxml")
    for items in soup.select("#sobi2CatListSymbols .sobi2SubcatsListItems a[title]"):
        if items.text=="Tutors":
            ilink = f"{urljoin(url,items.get('href'))}"
    return ilink

def get_links(tlink):
    linklist = []
    res = requests.get(tlink)
    soup = BeautifulSoup(res.text,"lxml")
    for item in soup.select(".sobi2ItemTitle a"):
        linklist.append(urljoin(url,item.get("href")))
    return linklist

if __name__ == '__main__':
    with open("task_scheduler.csv","a",newline="") as infile:
        writer = csv.writer(infile)
        item = get_info(url)
        for nlink in get_links(item):
            print(nlink)
            writer.writerow([nlink])

I've clearly specified the location of the ".bat" file below:

"C:\Users\WCS\AppData\Local\Programs\Python\Python36-32\test_schedule.bat"

Answer 1

Your code writes to the current working directory , because it opens a filename without a path. The location of the .bat file does not determine the current working directory of code being run.

You didn't specify a working directory; I'd expect that to be set with the Start in field in the scheduler, so the file is being created in the default location , %Windir%\\System32\\ .

Either set a directory in the Start in field for the CSV file to be created in, have your batch script use cd to move the current working directory to the right location, or use an absolute path when opening the file.

For example, if your batch script first runs cd /d %~dp0 then the working directory is changed to the directory of the batch script, and relative paths in Python are resolved against that location.