This is what I was thinking about with the prefix issue. I've encountered this issue like twice and rebooting my router ended up with a different prefix, so it seems more like my router just didn't get the new prefix and thus kept handing out the old prefix to everyone with SLAAC and thus wouldn't get routed right.