Actually I find the opposite. Python is easy for quick wins but from a maintenance, performance and reliability perspective it rapidly falls off a cliff. If your code propels your business then these concerns need to be considered up front.
Even off the shelf functionality doesnt have the same time/complexity properties as say c#, java or go resulting in a fairly basic algorithm reaching ridiculous time constraints very quickly.
As for CRUD type apps, you are probably right but its the bits that aren't CRUD which add the most value to your product.
"As for CRUD type apps, you are probably right but its the bits that aren't CRUD which add the most value to your product."
I'm sympathetic to your point, but I don't think this is necessarily true. What adds the most value to a product is determined by economics. It has nothing to do with whether an app is doing basic crud or nlp wizardry underneath, just whether it does something people find useful. There are tons of basic crud apps out there making people millionaires.
There are tons of basic crud apps out there making people millionaires.
Name three.
Economics drives both demand and supply. CRUD apps are easy to make; the market for most CRUD-only apps is heavily saturated with competitors, so the differentiating value of your application is not likely to be your ability to perform CRUD operations. Put it this way - if you come up with a CRUD app that does something staggeringly valuable, expect strong competitors in days not years. Your chance to make millions is small.
Perhaps my point wasn't quite described as well as it could have been. What I meant to say was:
What distinguishes your product is not how it stores or presents the data but what value it derives or extracts from that data.
Most CRUD apps are just a form of lock in which provide no value past storing your stuff in one place. That covers pretty much the entire enterprise software space and most trendy startups these days.
If there really are business-critical parts of the app that are too slow a good developer will drop down to C (as noted by others). That doesn't negate the reasons for using Python for the rest of the app.
Your mentions of maintenance and reliability in the opening sentence are unfounded. What makes a Python app less so than the same app written in any other language? Surely it comes down to the developer. A well-factored app written in Python (with tests) is going to be far more maintainable than a big ball of mud written in C. The choice of language has little to do with it.
I wouldn't drop down to C ever at application level. There is virtually no need to do this these days without introducing more risks from a security and memory perspective and the inevitable situation of "integration hell". I'd rather throw it in a JVM than drop to C. Then again if I started with a JVM I wouldn't have the problem to start with as it solves both ends of the problem. The CLR is equally applicable here. Go hits the mark too.
From my experience here dealing with a massive 100kloc python project. Python maintenance:
a) Horrible to refactor. I mean really horrible. You don't get everything right the first time and when it comes to fixing that it's hell.
b) no type safety or contracts which is hell when working with other developers.
c) indentation issues which are totally horrible at merge time as it introduces a shit ton more work to do.
Reliability:
a) Some algorithms don't scale the same way as they do in other languages due to the points the article raises. It gets to the point that for something in Java is O(N), it can approach O(N^2) due to all the dict fudging and copying which is not good.
b) It's slow anyway - I mean really slow. If something feels slow, it is slow. PyPy may fix this but (c) below causes an issue as well.
c) Optimisations aren't consistent across python versions (consider how crappy IO was in the first py3k drop).
I wouldn't use Python for a new project now. The only utility I see is quick disposable bits of glue.
Thanks for elaborating and I hope you also realise I was just using C as an example of another language (a type-safe one). I think a lot of us have encountered 'big ball of mud C code-bases' in our careers which is what tends to spring to mind for me when anyone is discussing code maintainability.
>> Horrible to refactor. I mean really horrible. You don't get everything right the first time and when it comes to fixing that it's hell
I think dynamic languages like Python and Ruby are an absolute joy to refactor. There is generally less code-junk to be worrying about when making refactorings. I realise this is subjective however, I think it's difficult to make a serious case that any one language is easier to refactor than another. Granted, the refactoring tools in IDEs for Ruby/Python etc. are much more limited than say C#.
I hope you won't mind if I don't get into the indentation or type safety arguments. They've been done to death! :)
I think we're in agreement that if you need something to be really fast you should use a different language. I just don't think that's a reason to not use Python for the rest of your app. And of course as others have pointed out the vast majority of us are not writing apps that require super fast algorithms, we're all far too busy doing the same CRUD stuff over and over right :)
I appreciate your insight on a 100kloc Python project though, I have to admit I've not encountered anything of that size. At work we have a much smaller codebase that has already gotten out-of-hand but I believe that's entirely due to the developer responsible, not the language.
The problem with refactoring dynamic languages is that it's very hard for static code checking to let you know when you've missed a spot.
For example, what functions take class Foo that should now expect class Bar? What functions called method Baz, that now should call Bat?
Static typing makes all of that trivial. You know with certainty, when you've hit all the right spots, because the code won't compile otherwise. Also, a lot of that refactoring can be done automatically through tooling. "Replace all calls of Foo.Baz() with Bar.Bat()". Trivial, done... and it's often impossible to do the same thing in dynamic languages. You have to rely on tests catching everything... and how many of those tests need to be updated now, too? What's testing your tests?
I love dynamic languages, don't get me wrong... but refactoring large code based is way easier to get correct in statically typed languages.
If you need to touch a lot of places for refactoring on a regular basis, that should tell you you're not doing something optimally. If you write your tests so they are dependent on internal details of the other code, that's a warning sign too. Add to that, that at least in my experience, my Ruby code bases tends to end up far smaller than my old equivalent C code, and the problem tends to be a lot smaller than one might imagine.
Yes, you need testing discipline, but you need that with static languages too - if you think you're ok just because the compiler didn't complain.... Well, that's just a false sense of security.
Even off the shelf functionality doesnt have the same time/complexity properties as say c#, java or go resulting in a fairly basic algorithm reaching ridiculous time constraints very quickly.
As for CRUD type apps, you are probably right but its the bits that aren't CRUD which add the most value to your product.