Pass parameters from one notebook to another

Question

I have two notebooks in python.

The first one checks if a file exits in the data lake. I want to return a Boolean from here and the filePath if it exits.

The next notebook will then uses these Params as in input. How is this possible?

Also could I use a condition IF in the pipeline to check the returned Boolean?

Kind of new to Azure ADF

Answer 1

One method to pass messages between separate python scripts or jupyter notebooks is to use the pyzmq library. Run pairserver in one notebook and pairclient in another. You will see messages being passed from one to the other. This does add an extra dependency to your code, but pyzmq is a mature package.

pairserver.ipynb

#!/usr/bin/python3
import zmq
import random
import time

port = '5556'
context = zmq.Context()
socket = context.socket(zmq.PAIR)
socket.bind('tcp://*:%s' % port)

while True:
    socket.send(b'Server message to client')
    msg = socket.recv()
    print(msg)
    time.sleep(1)

pairclient.ipynb

#!/usr/bin/python3
import zmq
import random
import sys
import time

port = '5556'
context = zmq.Context()
socket = context.socket(zmq.PAIR)
socket.connect("tcp://localhost:%s" % port)

while True:
    msg = socket.recv()
    print(msg)
    socket.send_string("client message to server")
    time.sleep(1)