Even if you compile your own version of Signal, will your friends do it too? Will your grandma/grandpa do it as well? It only takes one person in the chain to be compromised by using the "real" app and then all your efforts would be defeated because now your messages have been exposed by this other person unknowingly.
Do phones have trusted execution environments? I suppose you could require the recipient provide attestation that it's running the expected binary. Of course, this is pointless if the hardware manufacturer shares their root keys with the government.
The backdoored app will hopefully not be called Signal, since Signal themselves would never do this. I hope they own a trademark on it and could enforce it against anyone who would try to upload a backdoored version under their name.
People will use what is most convenient. If tomorrow Signal leaves the EU, WhatsApp will happily take its place and will happily enforce the scanning and everyone will just have to fall in line.
What good is it if you are the only one of your family who has the only "uncompromised" app on your phone? How will you talk to them? Any message you send will be scanned on the other end.
That also applies if you have friends overseas. Your friend from Japan/US will be compromised as well.
