Failing To Find Chef Community Cookbooks Using Chef 12 On AWS Opsworks

Question

The Problem

When running custom cookbooks on AWS OpsWorks a setup_failed status appears against the instance and the following in the failure log:

[2016-03-26T22:53:48+00:00] INFO: Started chef-zero at chefzero://localhost:8889 with repository at /var/chef
One version per cookbook
data_bags at /var/chef/runs/62832572-cb67-421a-8309-d831140d7b98/data_bags
nodes at /var/chef/runs/62832572-cb67-421a-8309-d831140d7b98/nodes
 
[2016-03-26T22:53:48+00:00] INFO: Forking chef instance to converge...
[2016-03-26T22:53:48+00:00] INFO: *** Chef 12.7.2 ***
[2016-03-26T22:53:48+00:00] INFO: Chef-client pid: 17842
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: chefzero://localhost:8889/nodes/test1.localdomain
[2016-03-26T22:53:49+00:00] INFO: Setting the run_list to ["recipe[my_cookbook::default]"] from CLI options
[2016-03-26T22:53:49+00:00] INFO: Run List is [recipe[my_cookbook::default]]
[2016-03-26T22:53:49+00:00] INFO: Run List expands to [my_cookbook::default]
[2016-03-26T22:53:49+00:00] INFO: Starting Chef Run for test1.localdomain
[2016-03-26T22:53:49+00:00] INFO: Running start handlers
[2016-03-26T22:53:49+00:00] INFO: Start handlers complete.
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: 
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 412 Precondition Failed: No such cookbook: apt
 
================================================================================
Error Resolving Cookbooks for Run List:
================================================================================
 
Missing Cookbooks:
------------------
No such cookbook: apt
 
Expanded Run List:
------------------
* my_cookbook::default

---

The OpsWorks stack is configured

• for Chef 12
• to use custom Chef cookbooks
• to use a Git repository for the custom cookbooks

The Custom Cookbook Repo

For the sake of finding the cause of the problem, a cookbook was created with a single recipe that runs apt-get update (which runs as expected under kitchen converge ). The files were created using chef generate cookbook my_cookbook :

.
└── my_cookbook
    ├── Berksfile
    ├── chefignore
    ├── metadata.rb
    ├── README.md
    ├── recipes
    │   └── default.rb
    ├── spec
    │   ├── spec_helper.rb
    │   └── unit
    │       └── recipes
    │           └── default_spec.rb
    └── test
        └── integration
            ├── default
            │   └── serverspec
            │       └── default_spec.rb
            └── helpers
                └── serverspec
                    └── spec_helper.rb

The only altered files are:

my_cookbook/Berksfile

source 'https://supermarket.chef.io'    
metadata    
cookbook 'apt'

my_cookbook/metadata.rb

...
depends 'apt', '~> 3.0.0'

my_cookbook/recipes/default.rb

include_recipe 'apt::default'

Also, a Berksfile was added to the root of the repo

source 'https://supermarket.chef.io'
cookbook 'apt'

---

The Current Workaround

The current workaround involves downloading the dependencies using berks vendor and then copying the downloaded cookbooks into the root of the repo ie:

.
├── Berksfile
├── apt
└── my_cookbook

Is there a better solution (using AWS OpsWorks with Chef 12) that does not require explicitly storing and distributing community cookbooks with custom cookbooks?

Answer 1

I have the same issue. It looks like it is supposed to use berk pack with s3 storage, but (as to me) it adds a other level of complexity (and is useful only with CI tools, where cookbook archive created by berkshelf would be convenient 'artifact'):

In Chef 12 Linux, Berkshelf is no longer installed on stack instances. Instead, we recommend that you use Berkshelf on a local development machine to package your cookbook dependencies locally. Then upload your package, with the dependencies included, to Amazon Simple Storage Service. Finally, modify your Chef 12 Linux stack to use the uploaded package as a cookbook source. For more information, see Packaging Cookbook Dependencies Locally.

From here .

Answer 2

I'm pretty sure this is how you are supposed to do it now. They removed a bunch of the automated berks integration between the Chef 11 and Chef 12 stacks for reasons I've never understood.

Answer 3

Here is the answer for this above matter.

check your metadatatab.rb file and check your version of chef, my case it should be 12.0 or upper.

second before execute your customcookbooks, update it and run the execute.

once you run the updatecookbook check /var/chef/cookbook folder and check your files is there or not.

and then run executecookbook.

enjoy.