Flask response character set

Question

I'm developing my API and currently I store one of my fields in spanish. Database is Postgres 9.5 (field type is varchar):

File Encoding         : utf-8

Text field:

"text" varchar(65536) COLLATE "default"

When i return the value from text, I use flask_sqlalchemy to get my data.

class Serializer(object):
        """
            Serialize information from database to a JSON Dictionary
        """

        def serialize(self):
            return {c: getattr(self, c) for c in inspect(self).attrs.keys()}

        @staticmethod
        def serialize_list(l):
            return [m.serialize() for m in l]


class AutoSerialize(object):
    """
        Mixin for retrieving public fields of model in json-compatible format'
    """
    __public__ = None

    def get_public(self, exclude=(), extra=()):
        "Returns model's PUBLIC data for jsonify"
        data = {}
        keys = self._sa_instance_state.attrs.items()
        public = self.__public__ + extra if self.__public__ else extra
        for k, field in keys:
            if public and k not in public: continue
            if k in exclude: continue
            value = self._serialize(field.value)
            if value:
                data[k] = value
        return data

    @classmethod
    def _serialize(cls, value, follow_fk=False):
        if type(value) in (datetime,):
            ret = value.isoformat()
        elif hasattr(value, '__iter__'):
            ret = []
            for v in value:
                ret.append(cls._serialize(v))
        elif AutoSerialize in value.__class__.__bases__:
            ret = value.get_public()
        else:
            ret = value

        return ret

My field in my Model is defined as follows and my class inherits Serializer and AutoSerialize :

 description = Column(String(65536), nullable=True)

This is how I return my values to API client:

 articles = Model.Bits.query.order_by(Model.Bits.publishedAt.desc()).limit(10).all()
            if articles:
                log.info('api() | %d Articles found ' % len(articles))
                response = []
                values = ['author', 'title', 'description', 'url', 'urlToImage', 'publishedAt']
                response = [{value: getattr(d, value) for value in values} for d in articles]
                return jsonify(articles=response, status='ok', source='my_source', sortBy='latest')

My response looks like this using curl:

 {
      "author": "Bros Lopez", 
      "description": "Spotify quiere ser m\u00e1s competitivo en su servicio de recomendaciones de contenido frente a marcas como Apple Music y Pandora. La empresa anunci\u00f3 la compra de la startup francesa Niland, la cual utiliza un sistema de inteligencia artificial para b\u00fasqueda y recomendaciones de contenido. Con 50 millones de usuarios activos Spotify quiere ser m\u00e1s rentable, pues a pesar de que el a\u00f1o pasado gener\u00f3 $3.1 mmdd en ventas, su margen bruto fue del 14%, pagando cerca de 2.7 mmdd en sellos discogr\u00e1ficos y editoriales. Por su parte, Pandora, unos de los principales competidores de Spotify, podr\u00eda ser adquirida por la empresa de radiodifusi\u00f3n SiriusXM, quien el a\u00f1o pasado le hizo una propuesta de compra por $3.4 mmdd. More Info", 
      "publishedAt": "Fri, 19 May 2017 20:00:00 GMT", 
      "title": "\u00bfPandora o Spotify?", 
      "url": "http://www.cnbc.com/2017/05/18/spotify-buys-niland-french-ai-music-startup.html", 
      "urlToImage": "https://ci4.googleusercontent.com/proxy/TWmEZRwlpPQrjs4HGZGx2041GryyquO7CjSR0oVBK-JUy4Xv3qHSiDow056iW8DV059chC93zFeXc4GVHKnzPpweUy-JzamK-l9pkW-Hgl1PnOun5s4XsE7K2NXBJljp-1Ltf5jyjfcn4j63Hv68FdFuqsw5UNTFBKkFug0=s0-d-e1-ft#https://gallery.mailchimp.com/f82949535ab2aab4bafde98f6/images/1f0dc47c-358b-4625-8744-105ffccfed98.jpg"
    }

Is the encoding correct? I tried different client and characters are displayed correctly, so not sure if it is up to the client to display info properly or server.

Answer 1

It is a client's job to parse such characters, which curl is obviously not doing "out-of-the-box". Depending on OS/shell/encoding you are using, there might be some ways (or the others) to pipe the response to some other command which would parse those characters or some similar approach.