Problem always is how to get packet back. The IPv4 host can get packet from somewhere. But if it simply does not have enough bits to represent the return address it cannot send packet back that actually arrives. Unless you do something like NAT that is get help form something on the way. And that something then has to keep track of whatever was done.