It's much easier to get molded plastic into the same shape repeatedly, than it is to get fabric stitched together exactly the same way every time.
As for the Xbox controllers, it could be a lot of differences. Microsoft does not feel the need to keep different runs exactly the same. They may be more willing to buy different types of plastic depending on what's cheapest this quarter.
Or it could be much simpler, maybe iPhones do feel different. How much time do you actually spend interacting with someone else's phone (Have you ever even actually touched another person's phone, or just their case?), vs how much time did you spend with your buddy's Xbox controllers?
As for the Xbox controllers, it could be a lot of differences. Microsoft does not feel the need to keep different runs exactly the same. They may be more willing to buy different types of plastic depending on what's cheapest this quarter.
Or it could be much simpler, maybe iPhones do feel different. How much time do you actually spend interacting with someone else's phone (Have you ever even actually touched another person's phone, or just their case?), vs how much time did you spend with your buddy's Xbox controllers?