AWS CDK CodePipeline : Synth step fails

Question

I am using AWS CDK CodePipeline construct to try to build a continuous deployment pipeline. It's new for me so really don't hesitate to fill-in any understanding gaps or misunderstangins on my side.

I have a very simple backend coded in Python that is hosted on GitHub Enterprise. I have created a connection to my repository, it works.

Now the Synth step fails because the step npx cdk synth which is identified in every CDK workshop about CDK pipelines apparently can't find the cdk.json file:

--app is required either in command-line, in cdk.json or in ~/.cdk.json

And sure enough, when I print what is inside the directory, the file is not there. You can check that in the logs I have linked below.

I would like to understand what I am doing wrong because it seems I don't do much different things compared to the tutorial.

Here is my CDK code for the pipeline:

import {Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { CodePipeline, CodePipelineSource, ShellStep } from 'aws-cdk-lib/pipelines';

import * as codepipeline_actions from 'aws-cdk-lib/aws-codepipeline-actions';
import { Artifact, Pipeline } from 'aws-cdk-lib/aws-codepipeline';


export class PipelineStack extends Stack {
    constructor(scope: Construct, id: string, props?: StackProps) {
        super(scope, id, props);

        const pipeline = new CodePipeline(this, 'Pipeline', {
            // The pipeline name
            pipelineName: 'MyPipeline',
      
             // How it will be built and synthesized
             synth: new ShellStep('Synth', {
               // Where the source can be found
               input: CodePipelineSource.connection('USERID/REPO-NAME', 'master', {
                connectionArn: 'arn:aws:codestar-connections:eu-central-1:XXXXXX:connection/YYYYY', // Created using the AWS console * });',
              }),
               
               
               // Install dependencies, build and run cdk synth
               commands: ['./synth_script.sh', 'ls', 'npx cdk synth']
             }),
          });

    }
};

Here is the content of the synth_script.sh that is hosted on the GitHub repository. Note that I have played with everything in the script, some parts outside of it to try to understand what is wrong:

#!/bin/bash
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
source $HOME/.poetry/env
poetry build
ls

and here are the logs showing the failure from the pipeline:

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:44 Waiting for agent ping

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:45 Waiting for DOWNLOAD_SOURCE

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:45 Phase is DOWNLOAD_SOURCE

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:45 CODEBUILD_SRC_DIR=/codebuild/output/src749056129/src

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:45 YAML location is /codebuild/readonly/buildspec.yml

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:45 Processing environment variables

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:45 No runtime version selected in buildspec.

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Moving to directory /codebuild/output/src749056129/src

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Configuring ssm agent with target id: codebuild:4c1d3f35-df16-4471-bb05-520a38b0e41d

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Successfully updated ssm agent configuration

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Registering with agent

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Phases found in YAML: 1

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 BUILD: 3 commands

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Phase context status code: Message:

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Entering phase INSTALL

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Phase complete: INSTALL State: SUCCEEDED

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Phase context status code: Message:

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Entering phase PRE_BUILD

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Phase complete: PRE_BUILD State: SUCCEEDED

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Phase context status code: Message:

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Entering phase BUILD

2022-01-21T12:00:51.637+01:00   [Container] 2022/01/21 11:00:47 Running command ./synth_script.sh

2022-01-21T12:00:59.693+01:00   Retrieving Poetry metadata

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   This installer is deprecated. Poetry versions installed using this script will not be able to use 'self update' command to upgrade to 1.2.0a1 or later.

2022-01-21T12:00:59.693+01:00   # Welcome to Poetry!

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   This will download and install the latest version of Poetry,

2022-01-21T12:00:59.693+01:00   a dependency and package manager for Python.

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   It will add the `poetry` command to Poetry's bin directory, located at:

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   $HOME/.poetry/bin

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   This path will then be added to your `PATH` environment variable by

2022-01-21T12:00:59.693+01:00   modifying the profile file located at:

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   $HOME/.profile

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   You can uninstall at any time by executing this script with the --uninstall option,

2022-01-21T12:00:59.693+01:00   and these changes will be reverted.

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   Installing version: 1.1.12

2022-01-21T12:00:59.693+01:00   - Downloading poetry-1.1.12-linux.tar.gz (66.70MB)

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   Poetry (1.1.12) is installed now. Great!

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`

2022-01-21T12:00:59.693+01:00   environment variable. Next time you log in this will be done

2022-01-21T12:00:59.693+01:00   automatically.

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:00:59.693+01:00   To configure your current shell run `source $HOME/.poetry/env`

2022-01-21T12:00:59.693+01:00   

2022-01-21T12:01:03.743+01:00   Creating virtualenv agile-budget-calculator-mVCiprTw-py3.9 in /root/.cache/pypoetry/virtualenvs

2022-01-21T12:01:05.774+01:00   Building agile_budget_calculator (0.1.0)

2022-01-21T12:01:05.774+01:00   - Building sdist

2022-01-21T12:01:05.774+01:00   - Built agile_budget_calculator-0.1.0.tar.gz

2022-01-21T12:01:05.774+01:00   - Building wheel

2022-01-21T12:01:05.774+01:00   - Built agile_budget_calculator-0.1.0-py3-none-any.whl

2022-01-21T12:01:05.774+01:00   README.rst

2022-01-21T12:01:05.774+01:00   dist

2022-01-21T12:01:05.774+01:00   poetry.lock

2022-01-21T12:01:05.774+01:00   pyproject.toml

2022-01-21T12:01:05.774+01:00   src

2022-01-21T12:01:05.774+01:00   synth_script.sh

2022-01-21T12:01:05.774+01:00   tests

2022-01-21T12:01:05.774+01:00   

2022-01-21T12:01:05.774+01:00   [Container] 2022/01/21 11:01:04 Running command ls

2022-01-21T12:01:05.774+01:00   README.rst

2022-01-21T12:01:05.774+01:00   dist

2022-01-21T12:01:05.774+01:00   poetry.lock

2022-01-21T12:01:05.774+01:00   pyproject.toml

2022-01-21T12:01:05.774+01:00   src

2022-01-21T12:01:05.774+01:00   synth_script.sh

2022-01-21T12:01:05.774+01:00   tests

2022-01-21T12:01:05.774+01:00   

2022-01-21T12:01:05.774+01:00   [Container] 2022/01/21 11:01:04 Running command npx cdk synth

2022-01-21T12:01:16.064+01:00   npx: installed 213 in 8.412s

2022-01-21T12:01:16.064+01:00   --app is required either in command-line, in cdk.json or in ~/.cdk.json

2022-01-21T12:01:18.395+01:00   

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Command did not exit successfully npx cdk synth exit status 1

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Phase complete: BUILD State: FAILED

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: npx cdk synth. Reason: exit status 1

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Entering phase POST_BUILD

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Phase complete: POST_BUILD State: SUCCEEDED

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Phase context status code: Message:

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Expanding base directory path: cdk.out

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Assembling file list

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Expanding cdk.out

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Skipping invalid file path cdk.out

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Phase complete: UPLOAD_ARTIFACTS State: FAILED

2022-01-21T12:01:18.395+01:00   [Container] 2022/01/21 11:01:16 Phase context status code: CLIENT_ERROR Message: no matching base directory path found for cdk.out

And also here are some links to the tutorials I am following:

• https://cdkworkshop.com/20-typescript/70-advanced-topics/200-pipelines.html
• https://aws.amazon.com/blogs/developer/cdk-pipelines-continuous-delivery-for-aws-cdk-applications/

I see that I use a ShellStep instead of a CodeBuildStep but I don't see why this would cause the error?

Answer 1

You have to be in the correct dir to execute cdk synth. The correct dir is the one which contains the cdk.json file.