是不是在DLL / C-Connect中禁用了指针类型检查，这样可以吗？

Question

After this somehow related question Why can't I pass an UninterpretedBytes to a void* thru DLL/C-Connect? where we saw that I could not pass a Smalltalk array of bits to a void * parameter, I further analyzed the method responsible for checking the compatibility of formal pointer description with effective object passed as argument, and I think that I discovered another questionable piece:

CPointerType>>coerceForArgument: anObject
    ...snip...
    (anObject isKindOf: self defaultDatumClass)
        ifTrue: [
            (referentType = anObject type referentType
                or: [(referentType isVoid
                    and: [anObject type referentType isConstant not])
                or: [anObject type isArray not
                    or: [anObject type baseArrayType = referentType]]])
            ifTrue: [^anObject asPointer]].
    ...snip...

It means the following:

1. It first checks if the argument is CDatum (a proxy to some C-formatted rawdata and associated CType).

2. If so, it checks whether the type is the same as the formal definition in external method prototype (self).

3. If not, it could be that the argument is void *, in which case any kind of pointer is accepted (it has been checked that it is a pointer in the code that I snipped), except if it is pointer on a const thing.
   There is a first discrepancy: it should check if the formal definition is const void * and accept any pointer on const in this case... But that does not matter much, we rarely have actual argument declared const.

4. If not, it checks if either not an array (for example, int foo[2]), or an array whose type matches (same base type and dimension).

So, if the formal definition is for example struct {int a; char *b} *foo struct {int a; char *b} *foo , and that I pass a double * bar , the type does not match, there is no const qualifier mismatch, and the parameter is not an array, conclusion: we can safely pass it without any further checking!

That's a kind of pointer aliasing. We do not have an optimizing compiler making any speculation about the absence of such aliasing in Smalltalk, so that won't be the source of undefined behaviour. It could be that we deliberately want to force this sort of dirty reinterpret_cast for obscure reasons (since we can explicitly cast a CDatum, I would prefer the explicit way).

BUT, it might be that we completely messed up and passed the wrong object, with wrong type, wrong dimension, and that the address foo->b in my example above will contain either some re-interpreted garbage if pointer is 32bits aligned, or be completely undefined on 64 bits machine (because beyond the sizeof double).

AC compiler would warn me for sure about the aliasing, and prevent production of artifact with -Wall -Werror.
What troubles me here is that I do not even get a warning...

Does it sound correct?

Answer 1

Short answer: it's not OK to correct this behavior, because some low level user interface stuff depends on it (event loop). We can't even introduce a Warning or anything.

Longer story: I tried to rewrite the whole method with double dispatching (ask anObject if compatible with formal CPointerType rather than testing every possible Object class with repeated isKindOf: ).

But when ommitting the disgracious pointer aliasing tolerance, it invariably screw my Macosx 8.3 image with tons of blank windows opening, and blocked uninterruptable UI...

After instrumenting, it appears that the event loop relies on it, and pass aString asNSString (which is transformed into utf16, but stored into a ByteArray and thus declared unsigned char *), to an Objective C method expecting an unsigned short *.

It's a case where the pointer aliasing is benign, as long as we pass the good bytes.

If I try and fix asNSString with a proper cast to unsigned short *, then the UI blocks (I don't know why, but it would require debugging at VM level).

Conclusion: it's true that some distinction such as (unsigned char *) vs (char *) can be germane and should better not be completely prohibited (whether char is signed or not is platform dependent, and not all libraries have cleanly defined APIs). Same goes with platform dependent wide character, we have conversion methods producing the good bytes, but not the good types. We could eventually make an exception for char * like we did for void * (before void * was introduced, char * was the way to do it anyway)... Right now, I have no good solution for this because of the event loop.