You're presenting a dichotomy. Alienate superiors by communicating erratically or without solutions; or be a willing accomplice to incompetence and fraud.
I don't understand your interpretation. The company knows that there are cheaper ways to accomplish the task, but they don't care. They've made the knowing acknowledgement to spend on a solution that costs more on the belief that they are extracting some value from the additional cost.
The key is that that value is not strictly technical. You can present a technical solution that is cheaper, but doesn't offer the non-technical value they derive from being a player on the "Big Data" scene. They can say "Yeah, we use our main guy's Perl script" or they can say "Yeah, we use Hadoop".
Is the value in that worth $99k per month? That's a subjective judgment for each company to make based on their specific circumstances.
Performance is really not something you can design from scratch. If you are using Hadoop for a 1GB job, it's likely that you have architectural bottlenecks that will prevent you from scaling to multiple-terabyte workloads anyway.
And if you overcomplicate things, you can easily get to a state where there are 2 guys that both half-understood the Hadoop setup, but both left for different startups. Complexity alone does not make things simpler.
Of course, you can provide the non-technical value of providing ~training~ makework and resume lines for 10 code monkeys and their managers, but that is not really value to the company.
a) get its engineers to give a talk at a Hadoop conference, resulting in marketing [logo shown prominently around the conference], PR, and recruitment gainz;
b) get articles published about how the company uses cutting edge technology to do new things and all the other CIOs and big shots better listen up, resulting in prestige, PR, and recruitment gainz; (this happened to a client in real life)
c) reasonably field interrogatory questions from other fad followers, whether they are investors, journalists, peers, or whomever. When asked "How is YourCorp using data science and Big Data?", being able to say "We have a team working with that" is much better than having to say "Our guy Bob says that's just a fad, so we don't really 'do that'". This is basically a PR gain, but it means that investors and clients will feel the company is cutting edge, instead of backward philistines who listen to Bob all the time.
I could go on but it's pretty boring.
The point is that business is all about the customer's perception of the company as something to which they want to give money. If the business does not appear to be following the trends, they will be substantially harmed, because people do not want to get involved with an outmoded business. Being perceived as the last to adopt a new technology looks bad.
