How to pack / unpack a hex string (high nibble first) with Elixir

Question

I was wondering how I would work with hex strings in Elixir. Specifically, I'm interested in converting from Hex to ASCII.

In Ruby, an implementation of this may be:

["001C7F616A8B002128C1A33E8100"].pack('H*').gsub(/[^[:print:]]/, '.')

How would I accomplish this task with Elixir? I have tried:

<<00, 01, C7, F6...>>

but this isn't a correct representation of the hex for a string. Thanks for your time and assistance!

So I've made some progress but am currently struggling with the recursion aspect of this.

This is my solution thus far:

defmodule ElixirNetworkTools do
  def decode(payload) do
    upper_payload = String.upcase payload
    case Base.decode16(upper_payload) do
      :error -> decode_with_nonprintable_characters(payload)
      {:ok, decoded_payload} -> decoded_payload
    end
    |> IO.write
  end

def decode_with_nonprintable_characters(payload) do
String.chunk(payload, ~r/\w{2}/)
|> Enum.each(fn(byte) ->
  case Base.decode16(byte) do
    :error -> '.'
    {:ok, decoded_payload} -> decoded_payload
      end
    end)
  end
end

Answer 1

Here is another solution to the problem. A couple things before we start:

• You can pass case: :mixed to Base.decode16/2 : Base.decode16(string, case: :mixed) , for this reason, you don't need do upcase before.

• If you are going to raise on an invalid string, don't bother checking, just call decode16 directly as it also checks the size.

This means we can start with:

decoded = Base.decode16!(string, case: :mixed)

Now you need to replace non-printable characters. Don't use String.printable?/1 because it is about UTF-8 and not ASCII. We need to implement our own function but what makes more sense: to raise or replace them? It seems it must be considered an error if someone send invalidate data? If that is the case:

def validate_ascii!(<<h, t::binary>>) when h <= 127 do
  validate_ascii!(t)
end

def validate_ascii!(<<>>) do
  true
end

def validate_ascii!(rest) do
  raise "invalid ascii on string starting at: #{rest}"
end

Alternatively you can just remove the last clause and it fail too.

Now we can put it together:

decoded = Base.decode16!(string, case: :mixed)
validate_ascii!(decoded)
decoded

EDIT: If you need to replace non-ascii by dots:

def keep_ascii(<<h, t::binary>>, acc) when h <= 127 do
  keep_ascii(t, acc <> <<h>>)
end

def keep_ascii(<<_, t::binary>>, acc) do
  keep_ascii(t, acc <> ".")
end

def keep_ascii(<<>>, acc) do
  acc
end

Answer 2

The solution ended up being as follows, though if there is a cleaner or better solution I would be very interested in knowing it.

defmodule ElixirNetworkTools do
  @doc """
  The decode function takes a hexadecimal payload, such as one generated
  by Snort, and returns the ASCII representation of the string.

  ## Example
  iex> ElixirNetworkTools.decode("436F6E74656E742D4C656E6774683A203132")
  {:ok, "Content-Length: 12"}
  """
  def decode(payload) do
    case _validate_length_of_snort(payload) do
      :error -> raise "Invalid length hex string. Must be even length. Exiting"
      _ -> nil
    end

    decoded = String.upcase(payload)
    |> _do_decode
    |> to_string

    {:ok, decoded}
  end

  @doc """
  Internal function used to manually process the hexadecimal payload,
  and builds a char list of the printable characters. If a character is
  not printable, we instead use periods.

  ## Example
  iex> ElixirNetworkTools._do_decode("436F6E74656E742D4C656E6774683A203132")
  ["Content-Length: 12"]
  """
  def _do_decode(payload) do
    Base.decode16!(payload)
    |> String.chunk(:printable)
    |> Enum.map(fn(chunk) ->
        case String.printable? chunk do
          true -> chunk
          _ -> "."
        end
    end)
  end

  @doc """
  Internal function used to validate the length of the hexadecimal payload.
  Hexadecimal strings should have an even number of characters.

  ## Example
  iex> ElixirNetworkTools._validate_length_of_snort("436F6E74656E742D4C656E6774683A203132")
  :ok
  """
  def _validate_length_of_snort(payload) do
    String.length(payload)
    |> rem(2)
    |> case do
      0 -> :ok
      _ -> :error
    end
  end
end