使用 docker-compose 在本地运行胶水容器

Question

I would like to run REPL shell (Pyspark) from the glue container using this command $ docker run -it -v ~/.aws:/home/glue_user/.aws -e AWS_PROFILE=$PROFILE_NAME -e DISABLE_SSL=true --rm -p 4040:4040 -p 18080:18080 --name glue_pyspark amazon/aws-glue-libs:glue_libs_3.0.0_image_01 pyspark mentioned in pyspark-glue-container but instead of running as docker run I would like to run as docker-compose .Hence created below compose file:

container_name: "glue_container"
image: amazon/aws-glue-libs:glue_libs_2.0.0_image_01
environment:
  DISABLE_SSL: "true"
ports:
  - 4040:40404
  - 18080:18080
command: pyspark
volumes:
  - ${PWD}/local_path_to_workspace:/home/glue_user/workspace/

but I'm not able to get the glue pyspark shell up and running? but the container is exiting with an error.

Answer 1

I'm guessing you got a "volume must be a mapping" error

Try this format

services:
  pyspark_repl:
    image: amazon/aws-glue-libs:glue_libs_2.0.0_image_01
    environment:
      DISABLE_SSL: "true"
    ports:
      - 4040:40404
      - 18080:18080
    command: pyspark
    volumes: 
      - ${PWD}:/home/glue_user/workspace/

Then run it like this

docker-compose run --rm pyspark_repl