Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Sysbench for MySQL 5.0, 5.1, 5.5, 5.6, 5.7 and 8 (smalldatum.blogspot.com)
66 points by tkyjonathan on June 16, 2017 | hide | past | favorite | 32 comments


It would be interesting to do this same comparison with MariaDB.

I wonder if that project has managed to continue progressing while MySql seems to have regressed.

It's super easy for all of us on HN to say "Oh look Oracle ruined another thing". But it would much more interesting to have quantified results to compare from.


Blog author here. I think that Oracle has been the best corporate owner for MySQL, much better than Sun and standalone MySQL.

I write these posts to document & market the problem and motivate upstream and my team to make things better. I don't think they are ruining MySQL.


As a Postgres user, something I find our community keeps hammering is MySQL's reluctance to do things safely. Be that fsyncing, or what, I'm never entirely sure.

I don't mean to further smear MySQL, just point out that this might not be a regression. It might be MySQL fixing bigger issues in default configurations. Spending 18% raw performance might be worth it.


Hmmm, consistently deteriorating performance over time is rather suspicious. A move by Oracle to kill off competition for their implementations? I can only speculate, though.

It seems odd, though. I guess it's all PostgreSQL all the time for me, from now on.


I think the author is specifically evaluating low-concurrency in-memory workloads here. The previous post describes why regressions for those workloads might be "not a surprise": https://smalldatum.blogspot.com/2017/05/the-history-of-low-c...


This context is key here. High concurrency workloads have not suffered this badly and in most cases have increased performance and that's specifically part of why. For many (or possibly even a good majority) of users, that's more important than low-concurrency.

Those blindly suggesting MariaDB or Percona is all well and good but naive. Percona generally follows MySQL upstream but makes their own little tweaks which could be (but are not always) fixed in MySQL upstream based on differing specific priorities.

MariaDB is highly diverged at this point and likely has a whole different set of problems and benefits.

Author of these posts is well known for a long (10+ years) history of working with MySQL at a deep level rivalling that of many of the developers, generally writes good stuff (though is not infallible :)


Yes, it seems that Oracle does care about performance in high-concurrency applications: https://www.mysql.com/why-mysql/benchmarks/

Even their own benchmarks show MySQL 5.7 dipping below previous versions at the low end of the concurrency scale.


Yes, context is everything when looking at benchmark results. I am showing the worst-case for this performance regression. Thanks for mentioning that. Note, that is my blog.

I also showed the impact for IO-bound workloads, and even there the regression is larger than I want. But not as bad as the in-memory workloads.


MariaDB is a thing.


Yeah, but why would I use that over PostgreSQL? I already know postgreSQL and it's also THE gold standard for SQL implementations. AFAIAC PostgreSQL is the obvious choice.


You would use it over PostgreSQL because it's a drop-in replacement for MySQL. PostgreSQL is not.

If you're not using MySQL in the first place, why would you care?


Because sometimes I DO use MySQL because a client requested it. At this point, I'll probably just explain to clients why pg is a much better alternative.


From a fresh build, I agree with you.

However, for MySQL in an existing application, the migration costs might make it not worth it. There can be a number of pernicious gotchas that aren't obvious at first. MariaDB is more or less set-and-forget.


tldr: mysql is getting worse for small, in memory workloads with low concurrency.

for the average medium to high traffic mysql install, i find 5.7 to be much faster than 5.6 or mariadb where concurrency and varying workloads prevail.

ymmv, do your own testing to see which db is a good platform for your specific workload. maybe it's mysql 5.6 maybe it's mariadb, maybe it's postgres.

for me mysql 5.7 brings much awaited niceness, saner defaults and even though not all workloads are better on it i think oracle is really trying to iterate. likely they will fix this regression and there would be little reason to use it over 5.6 or older.


It looks like overall things are getting worse as we "upgrade".


Oracle buys MySQL, slowly kills it by a thousand small cuts to performance...


Regression is expected with new features. Oracle has been a great owner for MySQL. I think we, including Oracle, can make progress on this problem.


Well, people have for years complained about features missing from MySQL–often without noticing that they're complaining about problems that were fixed when Bill Clinton was still president.

Adding features without incurring performance penalties is probably hard to pull off. FWIW we're still talking about 10000 queries/sec.


Is there any reason nowadays not to deploy one of the more performant forks of MySQL (e.g. Percona, MariaDB)?


Are they more performant? It would be great to see the comparison on the same benchmark.

One reason not to move to mariadb could be slight differences in schema syntax edge cases. But if you're starting from scratch, it doesn't matter.


Typically the forks include many performance tweaks over vanilla and so generally perform better. Here is an old test showing 5.5 differences in performance: http://vbtechsupport.com/657/2/

Percona's fork is also much closer to vanilla (than MariaDB), so there shouldn't be any issues even for existing databases. Unless maybe if the system/cluster set up is very complex so there is additional cost for migration. For new projects though, I can't think of any reason to use vanilla over one of the forks.


Would Oracle slowly degrade performance so they could sling their enterprise offerings better.... hmm. However, this is just going to kill MySQL altogether rather than actually achieve its implied goal. I guess you could say, competition always finds a way.


That's why there is MariaDB.


Has anyone run similar tests for MariaDB? That is my blog, but I have been too busy with upstream MySQL to test MariaDB.


WTF, MySQL is not doing performance regression testing on each commit?? WUT.


I find it unlikely that Oracle doesn't have extensive performance testing infrastructure.

If they have it, how can they miss this? Is it too specific?

If they haven't, did they throw it away after acquiring Sun? Or Sun never had it?


Or feature branch at least


In a world where Postgres exists, and Microsoft SQL Server Express is supported on Linux, what's the advantage of using MySQL?


Lots of reasons why I still use it:

- MySQL/MariaDB are much better tested for almost every OSS web app and framework I use. It may be the only database that is official and thoroughly supported. If I want to interoperate on the same DB across many applications, MySQL/MariaDB is the lingua franca.

- It's fast enough. The vast majority of applications never need to scale beyond a few hundred requests per second. Any database on nearly any hardware with sufficient memory will do this.

- Aurora. Scale up automagically (not infinitely, but pretty easily) when you do find yourself with an app that needs to scale a lot and fast.

- It's fine. Sure, PostgreSQL does a number of things better (though I'd wager some of the reasons MySQL is getting slower for some workloads is because it is now doing some of the things PostgreSQL has done for a while, and they require more resources; features have a cost). If I were weighing out the pros and cons in isolation, I'd pick PostgreSQL. But, I'm not. I'm building things on top of other things, and those other things were built with MySQL. when MySQL was faster than PostgreSQL I didn't choose it because of its speed (I used to use PostgreSQL for my company website, but migrated to MySQL about ten years ago).

MySQL/MariaDB may not be better, but they're good enough. MySQL and PostgreSQL have converged in terms of features and performance to be reasonable substitutes for each other.


> MySQL and PostgreSQL have converged in terms of features and performance

Is there CTEs in MySQL yet?

What about binary JSON and indexes on that?

How is implementing arrays and aggregate functions beyond simple arithmetic going?

Or maybe there is a way to write triggers in Python or JS if I want some performant number crunching with ACID guarantees?

Can DDL statements be rolled back, so I wouldn't have to pull my hair out during development after running migration I'm working on, if it happens to have a typo?



Inertia and being good enough at the jobs thrown at it.

Everyone for whom MySQL is not enough just move to Postgres and most open source ORM's support both so it's mostly a non issue. At a job I was told we would use Postgres very very shortly before release. All I had to do is change configuration settings and run a command, and that was 5 years ago.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: