Suppress stderr while running Mix Test in Elixir

Question

I'm working with several external processes in my application. During tests the stderr output of several of these is output inline with my test messages and results. I can do this:

mix test --trace 2> error.log

However, when I do, I lose all my lovely colors. Also some Elixir errors still appear, though not all (which is fine for me).

Is there a better way to suppress the errors of the external programs without affecting the mix output? Is it even a good idea?

Or should my tests not actually interact with the real command-line utilities? I ask because at that point I'm honestly no longer clear on what I'm testing.

UPDATE:

Below is a simplified function and test to illustrate the concept better.

The function:

@doc "Function takes a pre-routing rule as a string and adds it with iptables"
def addrule(pre_routing_rule)
  %Porcelain.Result{out: _output, status: status} = Porcelain.shell("sudo iptables -t nat -A #{pre_routing_rule}")
end

The test:

test "Removing a non-existent rule fails" do 
  Rules.clear
  assert {:error, :eiptables} == Rules.remove("PREROUTING -p tcp --dport 9080 -j DNAT --to-destination 192.168.1.3:9080")
end

This test passes perfectly. However inline with the test messages it also outputs iptables: No chain/target/match by that name. . The exact position of the message is also unpredictable and en masse these messages make test information very hard to read. Then I redirect stderr and I lose my colour-coding for some reason which also makes it hard to follow test results.

Answer 1

I think you should suppress these error messages at the point where you are calling the commandline utilities for example using the same method as in the question.

The lack of colors is connected to how Elixir detects ANSI :

[ANSI support] is by default false unless Elixir can detect during startup that both stdout and stderr are terminals.

Answer 2

I'd suggest in general that you decide how to deal with stderr in your system calls. What's happening is that the test framework catches stdout, but not stderr.

You can change how stderr is dealt with in the initial Porcelain call See

Porcelain API Docs