Error: Could not autoload puppet/provider/…: cannot load such file

Question

I have a puppet configuration that works on one Windows VM (Windows 2008 R2) using puppet apply. I copied the code to the same directory on a second VM (same OS) but puppet apply fails with this error:

Error: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip
Error: Could not autoload puppet/type/mirrored_directory: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip
Error: Could not autoload puppet/type/mirrored_directory: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip on node usb-tg
2008r2t2.xxxx.com
Error: Could not autoload puppet/type/mirrored_directory: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip on node usb-tg
2008r2t2.xxxx.com

Here is my source tree:

C:\PHX_DEPLOYER
└───puppet
    ├───manifests
    └───modules
        ├───dotnet
        │   ├───files
        │   ├───lib
        │   │   └───puppet
        │   │       └───parser
        │   │           └───functions
        │   └───manifests
        ├───mirrored_directory
        │   └───lib
        │       └───puppet
        │           ├───provider
        │           │   └───mirrored_directory
        │           │       └───zip
        │           └───type
        ├───tfview
        │   ├───files
        │   │   └───tf
        │   └───manifests
        └───tf_package
            └───manifests

I started a puppet cmd console running as admin. I manually set HOMEDRIVE=C: to fix one problem (our boot scripts sets it to H: drive). I set modulepath and hiera_config to match what I had on the other machine:

puppet config set modulepath c:/phx_deployer/puppet/modules
puppet config set hiera_config c:/phx_deployer/puppet/hiera.yaml

I also ran "puppet config print" and captured the output from both machines and used a diff program on it. The only settings that are different relate to the security certificate, IP address, and machine name.

From the message, it is clear that "puppet apply site.pp" is looking in the wrong place for my mirrored_directory defined type. It is looking in the standard puppet location, not my directory tree. What setting am I missing?

NOTE:

c:\>puppet --version
3.4.3 (Puppet Enterprise 3.2.3)
c:\>ruby --version
ruby 1.9.3p484 (2013-11-22) [i386-mingw32]

UPDATE:

The cause is this: My custom type provider code for my mirrored_directory type requires rubyzip. I declared a dependency on rubyzip package, but it never gets past the compile stage.

In nodes.pp:

package { 'rubyzip':
  ensure   => 'latest',
  provider => 'gem',
}

In the init.pp for tf_package:

...
  mirrored_directory { $package_dir:
    source       => "${staging_dir}/${specific_version_label}/${zip_filename}",
    ensure       => 'present',
    do_deletes   => $do_deletes,
    exclude      => $exclude,
    include      => $include,
    require      => [Package['rubyzip'],File[$package_dir]],
    prefix_depth => $prefix_depth
  }
...

I have the dependency on the package, but that is not enough. How does one architect a puppet solution so that you can make puppet install gems that are needed not by your application but by the puppet types themselves?

Update: Here is output from puppet apply:

c:\phx_deployer\puppet\manifests>puppet apply site.pp --debug
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/concat_basedir.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/custom_auth_conf.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/facter_dot_d.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/ip6tables_version.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/iptables_persistent_version.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/iptables_version.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/pe_build.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/pe_postgres_default_version.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/pe_puppetdb_server_status.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/pe_version.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/postgres_default_version.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/puppet_vardir.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/root_home.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/staging_http_get.rb
Info: Loading facts in C:/ProgramData/PuppetLabs/puppet/var/lib/facter/windows.rb
Debug: importing 'c:/phx_deployer/puppet/manifests/nodes.pp' in environment production
Debug: importing 'c:/phx_deployer/puppet/modules/dotnet/manifests/init.pp' in environment production
Debug: importing 'c:/phx_deployer/puppet/modules/dotnet/manifests/v4_5_1.pp' in environment production
Debug: Automatically imported dotnet::v4_5_1 from dotnet/v4_5_1 into production
Debug: hiera(): Hiera YAML backend starting
Debug: hiera(): Looking up dotnet::v4_5_1::version in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found dotnet::v4_5_1::version in deployer_windows
Debug: hiera(): Looking up dotnet::v4_5_1::installer in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found dotnet::v4_5_1::installer in deployer_windows
Debug: hiera(): Looking up dotnet::v4_5_1::temp_dir in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found dotnet::v4_5_1::temp_dir in deployer_windows
Debug: hiera(): Looking up phoenix::packages in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found phoenix::packages in deployer_windows
Debug: hiera(): Looking up phoenix::repository_dir in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found phoenix::repository_dir in deployer_windows
Debug: hiera(): Looking up phoenix::staging_dir in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found phoenix::staging_dir in deployer_windows
Debug: hiera(): Looking up phoenix::deployment_dir in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found phoenix::deployment_dir in deployer_windows
Debug: hiera(): Looking up phoenix::collection_name in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found phoenix::collection_name in deployer_windows
Debug: hiera(): Looking up phoenix::version_type in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found phoenix::version_type in deployer_windows
Debug: hiera(): Looking up phoenix::version_label in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found phoenix::version_label in deployer_windows
Debug: importing 'c:/phx_deployer/puppet/modules/tf_package/manifests/init.pp' in environment production
Debug: Automatically imported tf_package from tf_package into production
Debug: Failed to load library 'selinux' for feature 'selinux'
Debug: hiera(): Looking up tf_server in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found tf_server in deployer_windows
Debug: hiera(): Looking up tf_port in YAML backend
Debug: hiera(): Looking for data source deployer_USB-TG2008R2T1.ef.com
Debug: hiera(): Cannot find datafile c:/phx_deployer/puppet/deployer_USB-TG2008R2T1.ef.com.yaml, skipping
Debug: hiera(): Looking for data source deployer_windows
Debug: hiera(): Found tf_port in deployer_windows
Debug: importing 'c:/phx_deployer/puppet/modules/tfview/manifests/init.pp' in environment production
Debug: Automatically imported tfview from tfview into production
Error: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip
Error: Could not autoload puppet/type/mirrored_directory: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip
Error: Could not autoload puppet/type/mirrored_directory: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip on node usb-tg
2008r2t1.ef.com
Error: Could not autoload puppet/type/mirrored_directory: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip on node usb-tg
2008r2t1.ef.com

The first lines of my ruby module zipinstall.rb. require 'zip' needs gem rubyzip:

require 'rubygems'
require 'zip'

This code is in turn included in my provider code zip.rb:

# Resource Type: mirrored_directory
# Provider:      zip
#
# Provides the interface between the Puppet custom resource type 'mirrored_directory'
# and the ruby class 'ZipInstall::Unzipper'.
# This permits missing or changed files to be extracted from a Zip archive on a Windows system,
# while unchanged files are left alone.

require 'puppet/provider/mirrored_directory/zip/zipinstall'

Puppet::Type.type(:mirrored_directory).provide :zip do
  desc "Uses rubyzip to extract new or modified files from a zip archive on a Windows system."
  confine :osfamily => :windows
  defaultfor :osfamily => :windows

  include Puppet::Util::Warnings

  # The mirrored_directory resource exists iff the target directory exists.
  def exists?
    Puppet::FileSystem::File.exist?(@resource[:name])
  end

  # Assuming that the target directory is empty (or only contains excluded files),
  # extract all included files from the source archive and deposit them in the target directory
  # (or corresponding subdirectory).
  # Do not delete any files.
  def create
    Puppet::Util::Log.new(:level => :debug, :message => "create: About to unzip #{@resource[:source]}") 
    change_count = unzipper.extract(false)
    @property_hash[:ensure] = exists? ? :present : :absent
  end

  # Delete the contents of the target directory and subdirectories 
  # (other than those that are excluded).
  def destroy
    change_count = unzipper.delete
    @property_hash.clear
  end

  # Check if the files in the target directory match those in the source zip archive or not.
  def is_current
    Puppet::Util::Log.new(:level => :debug, :message => "is_current: About to examine #{@resource[:source]}") 
    !unzipper.has_differences?(false)
  end

  # Extract each file from the source archive that differs from the corresponding file
  # in the target directory. 
  # Optionally delete files found in the target directory but not in the source archive,
  # according to the parameter :do_deletes.
  def is_current=(value)
    unzipper_to_use = unzipper
    changes = unzipper_to_use.differences(false, false)
    count = unzipper_to_use.apply_changes(changes, @resource[:do_deletes])
    @property_hash[:is_current] = value
    0
  end

  # TODO: Search computer for all installed directories so that the installed 
  # mirrored_directory resource instances can be queried by puppet.
  def self.instances
    []
  end

  # Construct a new Unzipper, using these parameters supplied to the resource type:
  #   :source, :name, :exclude, :include
  # Supplies a logger that delegate to puppet's own logging.
  def unzipper
    # Puppet has numerous loggin levels, including:
    #    :crit, :err, :warning, :notice, :info, :debug
    logger = ->(level,message) {
      Puppet::Util::Log.new(:level => level, :message => message) 
    }
    ZipInstall::Unzipper.new(@resource[:source], @resource[:name], :prefix_depth => @resource[:prefix_depth], :exclude_files => @resource[:exclude], :include_files => @resource[:include], :logger => logger) 
  end

end

Answer 1

The output

Error: Could not autoload puppet/provider/mirrored_directory/zip: cannot load such file -- zip

seems to indicate that the zip utility is missing on the system, or cannot be found by Puppet.