'Encoding::UndefinedConversionError “\xE9” from ASCII-8BIT to UTF-8' when using some ruby gems that using file.open in ruby on rails

Question

What's I want?

I want to generate .docx file or .odt file from template file in Rails 3.2
I want to use Japanese in it.
In ubuntu server 12.04 & ruby 1.9.3p194 & rails 3.2.8

What's happen?

I tried gems 'docx-templater' and 'serenity'

ruby-docx-templater

https://github.com/jawspeak/ruby-docx-templater

1 sample works good
2 try to do the same in my rails app

in controller as is sample

def gen_docx
  input_file = './app/template/ExampleTemplate.docx'
  data = {
    :teacher => "Priya Vora",
    :building => "Building #14",
    :classroom => :'Rm 202',
    :district => "Washington County Public Schools",
    :senority => 12.25,
    :roster => [
      {:name => 'Sally', :age => 12, :attendence => '100%'},
      {:name => :Xiao, :age => 10, :attendence => '94%'},
      {:name => 'Bryan', :age => 13, :attendence => '100%'},
      {:name => 'Larry', :age => 11, :attendence => '90%'},
      {:name => 'Kumar', :age => 12, :attendence => '76%'},
      {:name => 'Amber', :age => 11, :attendence => '100%'},
      {:name => 'Isaiah', :age => 12, :attendence => '89%'},
      {:name => 'Omar', :age => 12, :attendence => '99%'},
      {:name => 'Xi', :age => 11, :attendence => '20%'},
      {:name => 'Noushin', :age => 12, :attendence => '100%'}
    ],
      :event_reports => [
        {:name => 'Science Museum Field Trip', :notes => 'PTA sponsored event. Spoke to Astronaut with HAM radio.'},
        {:name => 'Wilderness Center Retreat', :notes => '2 days hiking for charity:water fundraiser, $10,200 raised.'}
    ],
      :created_at => "11-12-03 02:01"
  }
  DocxTemplater::DocxCreator.new(input_file, data).generate_docx_file()
end

3 but the error raised
the error raised at following point in gem (docx_templater.rb 22)

File.open(file_name, 'w') { |f| f.write(buffer) }

serenity

https://github.com/kremso/serenity

1 sample works well
2 do the same in my rails app and works well
like this

#encoding: utf-8
require 'serenity'
class Showcase
  include Serenity::Generator

  Person = Struct.new(:name, :items)
  Item = Struct.new(:name, :usage)

  def generate_showcase
    @title = 'Serenity inventory'

    mals_items = [Item.new('Moses Brothers Self-Defense Engine Frontier Model B', 'Lock and load')]
    mal = Person.new('Malcolm Reynolds', mals_items)

    jaynes_items = [Item.new('Vera', 'Callahan full-bore auto-lock with a customized trigger, double cartridge and thorough gauge'),
                Item.new('Lux', 'Ratatata'),
                Item.new('Knife', 'Cut-throat')]
    jayne = Person.new('Jayne Cobb', jaynes_items)

    @crew = [mal, jayne]

    render_odt 'app/template/showcase.odt'
  end
end

3 I tried my template including Japanese but the error raised.
the error raised at following point in gem(template.rb 22)

    def process context
  tmpfiles = []
  Zip::ZipFile.open(@template) do |zipfile|
    %w(content.xml styles.xml).each do |xml_file|
      content = zipfile.read(xml_file)
      odteruby = OdtEruby.new(XmlReader.new(content))
      out = odteruby.evaluate(context)

      tmpfiles << (file = Tempfile.new("serenity"))
      file << out    #!!!! HERE !!!!
      file.close

      zipfile.replace(xml_file, file.path)
    end
  end
end

What I did?

• I found that 'serenity' have rspec test for Greek (UTF-8) in gem.I tried Japanese the same way. and the test passed!. so I thought the problem is not in gems in rails setting.

• add magic comment "#encoding: utf-8" to my controller or lib file

• confirm 'config.encoding = "utf-8"' in config/application.rb

• add following to config/enviroment.rb above "initialize!"

  Encoding.default_external = Encoding::UTF_8
  Encoding.default_internal = Encoding::UTF_8
  

• I don't use any database in my rails app.

But all is nothing to do with my case... any idea? I may not know the basic things ...

Answer 1

This is a little old, but I just ran into this same situation. In my case, it was a matter of setting the tempfile to binary mode prior to writing to it:

  tmpfiles << (file = Tempfile.new("serenity"))
  file.binmode
  file << out   
  file.close

Hope this helps