There are startups that in fact ONLY have government contracts. There are mechanisms that allow small startups to be awarded contracts without the need to compete with larger companies. The advantage in this approach is that founders don't need to go find venture funding, allowing them to retain more ownership of their company. This gives them the ability to have more to offer to new and talented employees. The disadvantage... well the government work slowly (which is counter to the silicon valley culture of rapid development, growth, and sell).