Breakage due to OS upgrades is one thing. But at least make sure that the software doesn't do anything obviously bad. For example, I was dealing with a "security product" (basically something that profiles activity on the OS looking for malware activity) that we were required to roll out, and come to find out it reliably corrupts the RPM package database on both CentOS 7 and Amazon AWS Linux 2, whenever the OS is rebooted (assuming the software is set to start on boot). Things like this are unacceptable.