Bioformats-Python error: 'ascii' codec can't encode character u'\xb5' when using OMEXML()

Question

I am trying to use bioformats in Python to read in a microscopy image (.lsm, .czi, .lif, you name it), print out the meta data, and display the image. ome = bf.OMEXML(md) gives me an error (below). I think it's talking about the information stored within md . It doesn't like that the information in md isn't all ASCII. But how do I overcome this problem? This is what I wrote:

import Tkinter as Tk, tkFileDialog
import os
import javabridge as jv
import bioformats as bf
import matplotlib.pyplot as plt
import numpy as np

jv.start_vm(class_path=bf.JARS, max_heap_size='12G')

User selects file to work with

#hiding root alllows file diaglog GUI to be shown without any other GUI elements
root = Tk.Tk()
root.withdraw()
file_full_path = tkFileDialog.askopenfilename()
filepath, filename = os.path.split(file_full_path)
os.chdir(os.path.dirname(file_full_path))

print('opening:  %s' %filename)
reader = bf.ImageReader(file_full_path)
md = bf.get_omexml_metadata(file_full_path)
ome = bf.OMEXML(md)

Put image in numpy array

raw_data = []
    for z in range(iome.Pixels.get_SizeZ()):
    raw_image = reader.read(z=z, series=0, rescale=False)
    raw_data.append(raw_image)
raw_data = np.array(raw_data)

Show wanted metadata

iome = ome.image(0) # e.g. first image
print(iome.get_Name())
print(iome.Pixels.get_SizeX())
print(iome.Pixels.get_SizeY())

Here's the error I get:

---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-22-a22c1dbbdd1e> in <module>()
     11 reader = bf.ImageReader(file_full_path)
     12 md = bf.get_omexml_metadata(file_full_path)
---> 13 ome = bf.OMEXML(md)

/anaconda/envs/env2_bioformats/lib/python2.7/site-packages/bioformats/omexml.pyc in __init__(self, xml)
    318         if isinstance(xml, str):
    319             xml = xml.encode("utf-8")
--> 320         self.dom = ElementTree.ElementTree(ElementTree.fromstring(xml))
    321 
    322         # determine OME namespaces

<string> in XML(text)

UnicodeEncodeError: 'ascii' codec can't encode character u'\xb5' in position 1623: ordinal not in range(128)

Here's a representative test image with proprietary microscopy format

Answer 1

Thank you for adding the sample image. That helped tremendously!

Let's first remove all the unnecessary Tkinter code until we get to a Minimal, Complete and Verifiable Example that allows us to reproduce your error message.

import javabridge as jv
import bioformats as bf

jv.start_vm(class_path=bf.JARS, max_heap_size='12G')

file_full_path = '/path/to/Cell1.lsm'

md = bf.get_omexml_metadata(file_full_path)

ome = bf.OMEXML(md)

jv.kill_vm()

We first get some warning messages about 3i SlideBook SlideBook6Reader library not found but we can apparently ignore that.

Your error message reads UnicodeEncodeError: 'ascii' codec can't encode character u'\\xb5' in position 1623: ordinal not in range(128) , so let's look what we can find around position 1623.

If you add print md after md = bf.get_omexml_metadata(file_full_path) , the whole xml with metadata is printed out. Let's zoom in:

>>> print md[1604:1627]
PhysicalSizeXUnit="µm"

So, the µ character is the culprit, it can't be encoded with the 'ascii' codec .

Looking back at the traceback:

/anaconda/envs/env2_bioformats/lib/python2.7/site-packages/bioformats/omexml.pyc in __init__(self, xml)
    318         if isinstance(xml, str):
    319             xml = xml.encode("utf-8")
--> 320         self.dom = ElementTree.ElementTree(ElementTree.fromstring(xml))
    321 
    322         # determine OME namespaces

We see that the in the lines before the error occurs, we encode our xml to utf-8 , that should solve our problem. So why doesn't it happen?

if we add print type(md) we get back <type 'unicode'> and not <type 'str'> as the code expected.. So this is a bug in omexml.py !

To solve this, do the following (you might need to be root);

• Go to /anaconda/envs/env2_bioformats/lib/python2.7/site-packages/bioformats/
• remove omexml.pyc
• in omexml.py change line 318 from isinstance(xml, str): to if isinstance(xml, basestring):

basestring is the superclass for str and unicode . It is used to test whether an object is an instance of str or unicode .

I wanted to file a bug for this, but it seems there is already an open issue .