Converting unsigned char * to hexstring

Question

Below code takes a hex string(every byte is represented as its corresponidng hex value) converts it to unsigned char * buffer and then converts back to hex string. This code is testing the conversion from unsigned char* buffer to hex string which I need to send over the network to a receiver process. I chose hex string as unsigned char can be in range of 0 to 255 and there is no printable character after 127. The below code just tells the portion that bugs me. Its in the comment.

#include <iostream>
#include <sstream>
#include <iomanip>

using namespace std;
// converts a hexstring to corresponding integer. i.e "c0" - > 192
int convertHexStringToInt(const string & hexString)
{
  stringstream geek;
  int x=0;

  geek << std::hex << hexString;
  geek >> x;

  return x;
}

// converts a complete hexstring to unsigned char * buffer
void convertHexStringToUnsignedCharBuffer(string hexString, unsigned char* 
hexBuffer)
{
  int i=0;
  while(hexString.length())
  {
    string hexStringPart = hexString.substr(0,2);
    hexString = hexString.substr(2);
    int hexStringOneByte = convertHexStringToInt (hexStringPart);
    hexBuffer[i] = static_cast<unsigned char>((hexStringOneByte & 0xFF)) ;
    i++;
  }
}

int main()
{
  //below hex string is a hex representation of a unsigned char * buffer.
  //this is generated by an excryption algorithm in unsigned char* format
  //I am converting it to hex string to make it printable for verification pupose.
  //and takes the hexstring as inpuit here to test the conversion logic.
  string inputHexString = "552027e33844dd7b71676b963c0b8e20";
  string outputHexString;
  stringstream geek;

  unsigned char * hexBuffer = new unsigned char[inputHexString.length()/2];
  convertHexStringToUnsignedCharBuffer(inputHexString, hexBuffer);

  for (int i=0;i<inputHexString.length()/2;i++)
  {
    geek <<std::hex << std::setw(2) << std::setfill('0')<<(0xFF&hexBuffer[i]); // this works
    //geek <<std::hex << std::setw(2) << std::setfill('0')<<(hexBuffer[i]); -- > this does not work
    // I am not able to figure out why I need to do the bit wise and operation with unsigned char "0xFF&hexBuffer[i]"
    // without this the conversion does not work for individual bytes having ascii values more than 127.
  }

  geek >> outputHexString;

  cout << "input hex string:  " << inputHexString<<endl;
  cout << "output hex string: " << outputHexString<<endl;
  if(0 == inputHexString.compare(outputHexString))
    cout<<"hex encoding successful"<<endl;
  else
    cout<<"hex encoding failed"<<endl;

  if(NULL != hexBuffer)
      delete[] hexBuffer;

  return 0;
}

// output
// can some one explain ? I am sure its something silly that I am missing.

Answer 1

The output of an unsigned char is like the output of a char which obviously does not what the OP expects.

I tested the following on coliru:

#include <iomanip>
#include <iostream>

int main()
{
  std::cout << "Output of (unsigned char)0xc0: "
    << std::hex << std::setw(2) << std::setfill('0') << (unsigned char)0xc0 << '\n';
  return 0;
}

and got:

Output of (unsigned char)0xc0: 0�

This is caused by the std::ostream::operator<<() which is chosen out of the available operators. I looked on cppreference

• operator<<(std::basic_ostream) and
• std::basic_ostream::operator<<

and found

template< class Traits >
basic_ostream<char,Traits>& operator<<( basic_ostream<char,Traits>& os,
                                        unsigned char ch );

in the former (with a little bit help from MM).

The OP suggested a fix: bit-wise And with 0xff which seemed to work. Checking this in coliru.com:

#include <iomanip>
#include <iostream>

int main()
{
  std::cout << "Output of (unsigned char)0xc0: "
    << std::hex << std::setw(2) << std::setfill('0') << (0xff & (unsigned char)0xc0) << '\n';
  return 0;
}

Output:

Output of (unsigned char)0xc0: c0

Really, this seems to work. Why?

0xff is an int constant (stricly speaking: an integer literal ) and has type int . Hence, the bit-wise And promotes (unsigned char)0xc0 to int as well, yields the result of type int , and hence, the std::ostream::operator<< for int is applied.

This is an option to solve this. I can provide another one – just converting the unsigned char to unsigned .

Where the promotion of unsigned char to int introduces a possible sign-bit extension (which is undesired in this case), this doesn't happen when unsigned char is converted to unsigned . The output stream operator for unsigned provides the intended output as well:

#include <iomanip>
#include <iostream>

int main()
{
  std::cout << "Output of (unsigned char)0xc0: "
    << std::hex << std::setw(2) << std::setfill('0') << (unsigned)(unsigned char)0xc0 << '\n';
  const unsigned char c = 0xc0;
  std::cout << "Output of unsigned char c = 0xc0: "
    << std::hex << std::setw(2) << std::setfill('0') << (unsigned)c << '\n';
  return 0;
}

Output:

Output of (unsigned char)0xc0: c0
Output of unsigned char c = 0xc0: c0

Live Demo on coliru

Answer 2

the C++20 way:

unsigned char* data = new unsigned char[]{ "Hello world\n\t\r\0" };
std::size_t data_size = sizeof("Hello world\n\t\r\0") - 1;

auto sp = std::span(data, data_size );
std::transform( sp.begin(), sp.end(),
                std::ostream_iterator<std::string>(std::cout),
                [](unsigned char c) -> std::string {
                    return std::format("{:02X}", int(c));
                });

or if you want to store result into string:

std::string result{};
result.reserve(size * 2 + 1);
std::transform( sp.begin(), sp.end(),
                std::back_inserter(result),
                [](unsigned char c) -> std::string {
                    return std::format("{:02X}", int(c));
                });

Output:
48656C6C6F20776F726C640A090D00