We do storage systems and use DPDK in the application, when the network IS the bottleneck it is worth it. Saturating two or three 400gbps NICs is possible with DPDK and the right architecture that makes the network be the bottleneck.

Once there will be a business around this and people will make money the businesses will maintain a lobby to keep doing it and even increase the operation.

I only use my common sense here, but it doesn't stop the production it just prevents the transport from the source to the destination through the area that was previously iron deficient and couldn't use the fully the other nutrient which passed on to another area.

It's easier to write the system's front end while paying little attention to the backend and "just" letting a local filesystem do a lot of the work for you, but it doesn't work well. The interesting question is if the result is also that the frontend-to-backend communication abstraction is good enough to replace the backend with a better solution. I'm not familiar enough with Ceph and BlueStore to have a conclusion on that.

I happen to work for a distributed file-system company, and while I don't do the filesystem part itself, the old saying "it takes software 10 years to mature" is so true in this domain.

I find that my understanding of different layers is usually not needed but is rarely important toy ability to solve thorny issues or figure out ways to do things or debug issues that others are not even trying.

I'm working in areas of systems programming where such knowledge and willingness to go deeper into the stack is helpful, even if rarely so.

I can't say I understand the kernel, I only scratch the surface of most of it and only dug deeper where I really needed to and only as deep as time allowed me and the need required though.

The advantage for text based interfaces is that they work over ssh/tmate over trans atlantic connections whereas the GUI tools would suffer greatly. It just works everywhere and if you learn it, it will be just as good as a gui debugger. Potentially even more ergonomic without hunting for options with a mouse.

Asynchronous IO with user space threads works wonders to get both the performance of async IO and the convenience of sequential programming.

My day job is working on a product that uses DPDK for a super high performance file system.

Your company already doesn't really care about you (for the vast majority of them at least), so there is a conflict of interest but you personally shouldn't take the side of the company.

There’s an element to that, for sure. But can it be generalised to a whole planet? I’m certain you have met people on your way who genuinely rooted for you. Companies are made out of people.

I do. To learn a code base it before tremendously, also to find what happened and why things were done it is off great help. I don't need it every day but I routinely do git archeology.