In general, assuming anything about data alignment is a bad practice that leads to very obscure errors. 
Taking the address of a packed member is dangerous since the reduced alignment of the pointee is lost.
If the pointer value is dereferenced, this can lead to memory alignment faults in some architectures.