Rails: Auto-reload gem files used in dummy app

Question

When developing a gem, I often use a dummy rails app that requires the gem in order to try out gem changes as I go. Also, I use the same dummy app for integration tests.

Usually, I have the gem in

~/rails/foo_gem

and the associated dummy app in:

~/rails/foo_gem/spec/dummy_app

With the zeitwerk code loader, how do I configure the dummy app to not only reload dummy-app ruby files on change, but also to pick up the changes to the gem files ? Otherwise, I would have to reload the dummy-app rails server for every change to the gem files while developing the gem.

# ~/rails/foo_gem/spec/dummy_app/config/environments/development.rb

config.cache_classes = false
config.eager_load = false

# TODO: Add ~/rails/foo_gem/lib to the list 
# of watched and auto-reloaded directories.

Did not work: config.autoload_paths

# ~/rails/foo_gem/spec/dummy_app/config/environments/development.rb

gem_root_path = Pathname.new(File.expand_path(Rails.root.join("../..")))
config.autoload_paths << gem_root_path.join("lib")

I've tried to add the gem to the autoload paths, but the code is not reloaded on file-system changes.

Did not work: Zeitwerk::Loader with enable_reloading

# ~/rails/foo_gem/spec/dummy_app/config/environments/development.rb

gem_root_path = Pathname.new(File.expand_path(Rails.root.join("../..")))
gem_loader = Zeitwerk::Loader.new
gem_loader.push_dir gem_root_path.join("lib")
gem_loader.enable_reloading
gem_loader.log!
gem_loader.setup

Adding a separate zeitwerk loader does not help; the loader does not come with a file-system watcher as far as I have understood it; so one needs to call gem_loader.reload in order to reload the gem classes.

Did not work: Zeitwerk::Loader#reload with require

If the gem's files are required within the gem, eg

# ~/rails/foo_gem/lib/foo_gem.rb

require 'foo_gem/bar`

then the file ~/rails/foo_gem/lib/foo_gem/bar.rb is ignored by the Zeitwerk::Loader . Calling gem_loader.reload does not reload this file.

Did not work: Zeitwerk::Loader#reload with zeitwerk loader for gem files

If the gem's files are not required manually, but instead a different zeitwerk loader is used, eg

# ~/rails/foo_gem/lib/foo_gem.rb

require "zeitwerk"
loader = Zeitwerk::Loader.new
loader.push_dir(__dir__)
loader.setup

then the directory ~/rails/foo_gem/lib is managed by two separate zeitwerk loaders: the loader in foo_gem.rb and the gem_loader in development.rb . This is apparently not allowed by zeitwerk, which complains with a Zeitwerk::Error :

loader... wants to manage directory ~/rails/foo_gem/lib, which is already managed by...

Did not work: ActiveSupport::FileUpdateChecker

# ~/rails/foo_gem/spec/dummy_app/config/environments/development.rb

gem_root_path = Pathname.new(File.expand_path(Rails.root.join("../..")))
gem_loader = Zeitwerk::Loader.new
gem_loader.push_dir gem_root_path.join("lib")
gem_loader.enable_reloading
gem_loader.log!
gem_loader.setup

gem_files = gem_root_path.glob("lib/**/*.rb")
gem_update_checker = ActiveSupport::FileUpdateChecker.new(gem_files) do
  gem_loader.reload  # This line is never executed
end
ActiveSupport::Reloader.to_prepare do
  gem_update_checker.execute_if_updated
end

I've tried to use the ActiveSupport::FileUpdateChecker to watch for changes, but, at least with my dockerized setup, the block to reload the code is never executed.

Answer 1

To reload the gem code on file-system changes, I needed to do three steps:

1. Unregister the zeitwerk loader defined in foo_gem.rb that handles loading the different gem files.
2. Define a new zeitwerk loader in the development.rb of the dummy app that is configured with enable_reloading .
3. Setup a file-system watcher and trigger a reload when a gem file changes.

I'm sure there is a simpler and cleaner solution. Feel free to post it as a separate answer. I'll post my solution here, but do consider it a workaround.

Zeitwerk::Loader in foo_gem.rb

# ~/rails/foo_gem/lib/foo_gem.rb

# require 'foo_gem/bar`  # Did not work. Instead:

require "zeitwerk"
loader = Zeitwerk::Loader.new
loader.push_dir File.join(__dir__)
loader.tag = "foo_gem"
loader.setup

Note:

• In the past, I've just loaded all ruby files of the gem with require from a kind of index file called just like the gem, here: foo_gem.rb . This does not work here, because zeitwerk appears to ignore files that have previously been loaded with require . Instead I needed to create a separate zeitwerk loader for the gem.
• This loader has no enable_reloading because otherwise, reloading would be enabled for this gem whenever using the gem, not just while developing the gem.
• I have given the loader a tag , which allows to find this loader later in the Zeitwerk::Registry in order to un-register it.

Zeitwerk::Loader in development.rb of the dummy app

# ~/rails/foo_gem/spec/dummy_app/config/environments/development.rb

# 1. Unregister the zeitwerk loader defined in foo_gem.rb that handles loading
#    the different gem files.
#
Zeitwerk::Registry.loaders.detect { |l| l.tag == "foo_gem" }.unregister
Object.send(:remove_const, :FooGem)

# 2. Define a new zeitwerk loader in the development.rb of the dummy app
#    that is configured with enable_reloading.
#
gem_root_path = Pathname.new(File.expand_path(Rails.root.join("../..")))
gem_loader = Zeitwerk::Loader.new
gem_loader.push_dir gem_root_path.join("lib")
gem_loader.enable_reloading
gem_loader.setup

# 3. Setup a file-system watcher and trigger a reload when a gem file changes.
#
Listen.to gem_root_path.join("lib"), only: /\.rb$/ do
  gem_loader.reload
end.start

Note:

• Zeitwerk does not allow two loaders managing the same files. Therefore, I need to unregister the previously defined loader tagged "foo_gem" .
• The new loader used in the dummy app has enable_reloading . Therefore, when using the dummy app with rails server , rails console , or when running the specs, the gem files can be reloaded.
• The gem files are not automatically reloaded by zeitwerk. One needs a file-system watcher to trigger the reload on file-system changes. I did not manage to get the ActiveSupport::FileUpdateChecker working. Instead, I've used the listen gem as file-system watcher.

With this setup, when using the rails server , the rails console of the dummy app or integration tests using the dummy app, gem files are reloaded after being edited, which means that one does no longer need to restart the rails server to pick up the changes.

References

• Zeitwerk::Loader
• Zeitwerk::Registry
• https://github.com/fxn/zeitwerk
• https://guides.rubyonrails.org/classic_to_zeitwerk_howto.html
• https://guides.rubyonrails.org/autoloading_and_reloading_constants.html

Answer 2

Auto reloading with Zeitwerk should just work if you run your Rails server in the root of your engine, instead of in the /test/dummy directory, like so:

# This:
/Users/lewis/work/myengine $ rails server

# NOT this:
/Users/lewis/work/myengine/test/dummy $ rails server

# NOT this:
/Users/lewis/work/myengine $ test/dummy/bin/rails server