As a tech guy who found an interest in design and ancillary fields recently, I am curious to know more. I assume leather, merino wool, cashmere do provide extra value. But other than that I have no knowledge. Eg why would 500 pants be better?
Material is not just about quality, but rarity or uniqueness. For example, japanese denim can get very expensive in part because it's very low volume. For dress pants, it might be a particularly interesting fabric.
A lot of more expensive pants also have interesting designs or proportions that are very unique or hard to find elsewhere. There is a lot of cool stuff you can get for under $500 USD though, that is still pretty expensive.
I have 2 pairs of pants that cost over $500. Both of them use technical fabrics (Schoeller Dryskin and Stotz EtaProof), have complex patterns (asymmetrical, articulated, etc.), lots of hardware (Riri zippers, magnetic pocket closures, Cobrax snaps), and can be ordered in custom sizing. They also have no text / logos anywhere on the pants. One pair is garment dyed as a complete unit after sewing to give a unique effect that's more interesting and has more "depth" compared to a flat, consistent color.