How to mock an aws library GlueContext using python unittest

Question

unable to mock GlueContext from aws glue

I have these lines of code

sc = SparkContext()
gluecontext = GlueContext(sc)
spark = gluecontext.spark_session

How to mock gluecontext using python unittest?

How to mock in a python unittest a library not installed locally?

Answer 1

One way is to download Glue libs and Spark to the root of your project (or configure as you like)

wget https://github.com/awslabs/aws-glue-libs/archive/glue-1.0.zip
wget https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-1.0/spark-2.4.3-bin-hadoop2.8.tgz
unzip glue-1.0.zip -d $PROJECT_ROOT
tar -xf spark-2.4.3-bin-hadoop2.8.tgz -C $PROJECT_ROOT
export SPARK_HOME=$PROJECT_ROOT/spark-2.4.3-bin-spark-2.4.3-bin-hadoop2.8

Then simply mock gluecontext

from mock import patch
class Test(unittest.TestCase):
    @patch('awsglue.context.GlueContext')
    @patch('awsglue.utils.getResolvedOptions', side_effect=mock_get_resolved_options)
    def test_method(self, mock_resolve_options, mock_glue_context):
        <your code>

Submit tests locally

$PROJECT_ROOT/aws-glue-libs-glue-1.0/bin/gluepytest $PROJECT_ROOT/tests/