How to remove all special characters in Linux text

Question

How to remove the special characters shown as blue color in the picture 1 like: ^M, ^A, ^@, ^[. In my understanding, ^M is a windows newline character, I can use sed -i '/^M//g' to remove it, but it doesn't work to remove others. The command dos2unix doesn't work, neither. Are there exist any ways that I can use to remove them both?

Answer 1

Remove everything except the printable characters (character class [:print:] ), with sed :

sed $'s/[^[:print:]\t]//g' file.txt

[:print:] includes:

• [:alnum:] (alpha-numerics)
• [:punct:] (punctuations)
• space

The ANSI C quoting ( $'' ) is used for interpreting \\t as literal tab inside $'' (in bash and alike).

Answer 2

要确保命令在Sed中使用有限的范围，强制使用“C”（POSIX）字符分类以避免使用非ASCII字符的不可预测的行为：

LC_ALL=C sed 's/[^[:blank:][:print:]]//g' file.txt

Answer 3

Try running below command on linux command prompt

Option - 1: (If dos2unix command is installed on Linux machine)

dos2unix sample_file.txt

Option - 2:

cat sample_file.txt | tr -d '\015' > new_sample_file.txt

Answer 4

Try this inside vi or vim:

[in ESC mode] type: :%s/^M//g

or:

sed -e "s/^M//" filename > newfilename

Important : To enter ^M, type CTRL-V, then CTRL-M