Pointer casting problem with struct array member

Question

I've run across this source in a legacy code base and I don't really know why exactly it behaves the way it does.

In the following code, the pData struct member either contains the data or a pointer to the real data in shared memory. The message is sent using IPC ( msgsnd() and msgrcv() ). Using the pointer casts (that are currently commented out), it fails using GCC 4.4.1 on an ARM target, the member uLen gets modified. When using memcpy() and everything works as expected. I can't really see what is wrong with the pointer casting. What is wrong here?

typedef struct {
    long mtype;
    unsigned short uRespQueue;
    unsigned short uID;
    unsigned short uLen;
    unsigned char pData[8000];
} message_t;

// changing the pointer in the struct
{
    unsigned char *pData = <some_pointer>;
#if 0
    *((unsigned int *)pMessage->pData) = (unsigned int)pData;
#else
    memcpy(pMessage->pData, &pData, sizeof(unsigned int));
#endif
}

// getting the pointer out
{
#if 0
    unsigned char *pData; (unsigned char *)(*((unsigned int *)pMessage->pData));
#else
    unsigned char *pData;
    memcpy(&pData, pMessage->pData, sizeof(int));
#endif
}

Answer 1

I suspect it's an alignment problem and either GCC or the processor is trying to compensate. The structure is defined as:

typedef struct {
    long mtype;
    unsigned short uRespQueue;
    unsigned short uID;
    unsigned short uLen;
    unsigned char pData[8000];
} message_t;

Assuming normal alignment restrictions and a 32-bit processor, the offsets of each field are:

mtype         0   (alignment 4)
uRespQueue    4   (alignment 2)
uID           6   (alignment 2)
uLen          8   (alignment 2)
pData         10  (alignment 1)

On all but the most recent versions of the ARM processor, memory access must be aligned on the ARM processor and with the casting:

*((unsigned int *)pMessage->pData) = (unsigned int)pData;

you are attempting to write a 32-bit value on a misaligned address. To correct the alignment, the address appears to have truncated the LSB's of the address to have the proper alignment. Doing so happened to overlap with the uLen field causing the problem.

To be able to handle this correctly, you need to make sure that you write the value to a properly aligned address. Either offset the pointer to align it or make sure pData is aligned to be able to handle 32-bit data. I would redefine the structure to align the pData member for 32-bit access.

typedef struct {
    long mtype;
    unsigned short uRespQueue;
    unsigned short uID;
    unsigned short uLen;
    union { /* this will add 2-bytes of padding */
        unsigned char *pData;
        unsigned char  rgData[8000];
    };
} message_t;

The structure should still occupy the same amount of bytes since it has a 4-byte alignment due to the mtype field.

Then you should be able to access the pointer:

unsigned char *pData = ...;
/* setting the pointer */
pMessage->pData = pData;

/* getting the pointer */
pData = pMessage->pData;

Answer 2

The point here is the code "int header = ( ((int )(txUserPtr) - 4))" Illustration of UserTypes and struct pointer casting is great of help!

typedef union UserTypes
{
    SAUser           AUser;
    BUser            BUser;
    SCUser           CUser;
    SDUser           DUser;
} UserTypes;

typedef struct AUser
{
    int              userId;
    int              dbIndex;
    ChannelType      ChanType;
 } AUser;
typedef struct AUser
{
    int              userId;
    int              dbIndex;
    ChannelType      ChanType;
 } AUser;

typedef struct BUser
{
    int              userId;
    int              dbIndex;
    ChannelType      ChanType;
 } BUser;

typedef struct CUser
{
    int              userId;
    int              dbIndex;
    ChannelType      ChanType;
 } CUser;

typedef struct DUser
{
    int              userId;
    int              dbIndex;
    ChannelType      ChanType;
 } DUser;

//this is the function I want to test

void Fun(UserTypes * txUserPtr)
{

   int header = (*((int*)(txUserPtr) - 4));

   //the problem is here
   //how should i set incoming pointer "txUserPtr" so that 
   //Fun() would skip following lines.
   // I don't want to execute error()

        if((header & 0xFF000000) != (int)0xAA000000)
        {
            error("sth error\n");
        }
   /*the following is the rest */ 
}

Answer 3

That is a very nasty thing to do (the thing that's compiled out). You're trying basically to hack the code, and instead of using the data copy in the message (in the provided 8000 bytes for it), you try to put a pointer, and pass it through IPC.

The main issue is sharing memory between processes. Who knows what happens to that pointer after you send it? Who knows what happens to the data it points to? That's a very bad habbit to send out a pointer to data that is not under your control (ie: not protected/properly shared).

Another thing that might happen, and is probably what you're actually talking about, is the alignment. The array is of char 's, the previous member in the struct is short , the compiler might attempt packing them. Recasting char[] to int * means that you take memory area and represent it as something else, without telling the compiler. You're stomping over the uLen by the cast.

memcopy is the proper way to do it.