The lib and classes folders are staged and deployed with IntelliJ but gcloud command does not include them

Question

I have a Java 8 Spring Boot GAE Standard App and I use IntelliJ IDE (Cloud Code plugins for GAE applications).

I use AppEngine-web.xml.

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
    <threadsafe>true</threadsafe>
    <sessions-enabled>true</sessions-enabled>
    <runtime>java8</runtime>
    <application>***</application>
    <service>***</service>
    <version>***</version>
    <url-stream-handler>urlfetch</url-stream-handler>

    <system-properties>
      <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
    </system-properties>

    <async-session-persistence enabled="true" />
    <instance-class>F2</instance-class>

    <resource-files>
        <include path="/**.json" />
    </resource-files>

    <use-google-connector-j>true</use-google-connector-j>
    <env-variables>
        <env-var name="GOOGLE_APPLICATION_CREDENTIALS"
                 value="WEB-INF/***.json" />
    </env-variables>
</appengine-web-app>

When I deploy my GAE app using IntelliJ, these folders are generated in my staging, and everything works well;

Here is a snapshot of my Eun COnfiguration on IntelliJ

The issue arises when I deploy it using gcloud command:

gcloud app deploy src/main/webapp/WEB-INF/appengine-web.xml --version=something --promote --stop-previous-version --verbosity=debug

Here is the generated staging folder:

I have .gitignore file in my app root; I removed that to checkif it is creating the issue and no luck and I also did the same with .cloudignore .

gcloud deploy skips files and the output is like:

DEBUG: Loading runtimes experiment config from [gs://runtime-builders/experiments.yaml]
INFO: Reading [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006BB8988>]
DEBUG:
Traceback (most recent call last):
  File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\runtime_builders.py", line 269, in _Read
    with contextlib.closing(storage_client.ReadObject(object_)) as f:
  File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\storage\storage_api.py", line 317, in ReadObject
    object_=object_ref, err=http_exc.HttpException(err)))
BadFileException: Could not read [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006BB8988>]. Please retry: HTTPError 404: No such object: runtime-builders/experiments.yaml
DEBUG: Experiment config file could not be read. This error is informational, and does not cause a deployment to fail. Reason: Unable to read the runtimes experiment config: [gs://runtime-builders/experiments.yaml], error: Could not read [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006BB8988>]. Please retry: HTTPError 404: No such object: runtime-builders/experiments.yaml
Traceback (most recent call last):
  File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\surface\app\deploy.py", line 146, in _ServerSideExperimentEnabled
    runtimes_builder_root)
  File "C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\runtime_builders.py", line 524, in LoadFromURI
    .format(uri, e))
ExperimentsError: Unable to read the runtimes experiment config: [gs://runtime-builders/experiments.yaml], error: Could not read [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006BB8988>]. Please retry: HTTPError 404: No such object: runtime-builders/experiments.yaml
WARNING: <application> and <version> elements in `appengine-web.xml` are not respected
INFO: Executing staging command: [C:\Program Files\Common Files\Oracle\Java\javapath\java.exe -classpath C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\java\lib\appengine-tools-api.jar com.google.appengine.tools.admin.AppCfg --enable_new_staging_defaults stage E:***\src\main\webapp c:\users\***\appdata\local\temp\tmp1nhtxn\tmpoizcfc]


DEBUG: Executing command: [u'C:\\Program Files\\Common Files\\Oracle\\Java\\javapath\\java.exe', u'-classpath', u'C:\\Program Files (x86)\\Google\\Cloud SDK\\google-cloud-sdk\\platform\\google_appengine\\google\\appengine\\tools\\java\\lib\\appengine-tools-api.jar', u'com.google.appengine.tools.admin.AppCfg', u'--enable_new_staging_defaults', u'stage', u'E:***\\src\\main\\webapp', 'c:\\users\\***\\appdata\\local\\temp\\tmp1nhtxn\\tmpoizcfc']
INFO: ------------------------------------ STDOUT ------------------------------------
Reading application configuration data...


Beginning interaction for module ***...
0% Scanning for jsp files.
2021-12-22 15:11:47.738:INFO::main: Logging initialized @167ms to org.eclipse.jetty.util.log.StdErrLog
2021-12-22 15:11:48.115:INFO:oeja.AnnotationConfiguration:main: Scanning elapsed time=0ms
0% Generated git repository information file.
Success.
Temporary staging for module *** directory left in C:\Users\***\AppData\Local\Temp\tmp1nhtxn\tmpoizcfc
------------------------------------ STDERR ------------------------------------
2021-12-22 15:11:47.221:INFO::main: Logging initialized @725ms to org.eclipse.jetty.util.log.StdErrLog
--------------------------------------------------------------------------------

Answer 1

I've managed to replicate this similar issue on a same project. This is because Java 8 may be considered a legacy version, but best practice still applies and that is to use App Engine Maven Plugin when submitting deployments. Full documentation could be found through this link .

I've changed the gcloud command:

gcloud app deploy src/main/webapp/WEB-INF/appengine-web.xml 
--version=something --promote --stop-previous-version --verbosity=debug

To this:

mvn clean package appengine:deploy

This process would take some time as it would clean and reinstall packages before deploying your project.

Here's a Github link similar to your project that you can try on your end as this one works for me.