How to check in linux shell encoding of string already generated by Python script

Question

I run a Python script, that generates a string and then execute a shell script using that string. I want to check the encoding of that string using linux shell but without writing that string in file (disk operations runs slowly). Is it possible to check an encoding of string in Linux (Ubuntu) using only RAM? Something like:

check-encoding 'My string with random encoding'

Python check encoding script is slow too.

Answer 1

Try file utility. You can pass any string as file argument to file by using echo piped to utility with - option (many commands use a hyphen (-) in place of a filename as an argument to indicate when the input should come from stdin rather than a file):

:~  $ echo "test" | file -i -
/dev/stdin: text/plain; charset=us-ascii

:~  $ echo "тест" | file -i -
/dev/stdin: text/plain; charset=utf-8

with pipe to sed:

:~  $ echo "тест" | file -i - | sed 's/.*charset=\(.*\)/\1/'
utf-8

or to awk (you can mix it of course):

:~  $ echo "тест" | file -i - | awk '{ print $3 }'
charset=utf-8

---

also you can use python chardet module. Chardet comes with a command-line script which reports on the encodings of one or more files. Just install it with:

pip install chardet

and use with pipe from echo:

:~  $ echo "тест" | chardetect
<stdin>: utf-8 with confidence 0.938125