It seems like a lot of the discussion surrounding DOD that gets popular interest is centered on a small set of patterns that you can apply. And the implication that DOD is the application of these patterns usually follows.
Taking this article as an example, it frames DOD as an optimization technique and explicitly states that these patterns are the main concepts of DOD.
But while these patterns are interesting and often show up in data-oriented designs, they are not themselves foundational to data-oriented design.
This is interesting to me because it seems to obviously be missing something. If the article went through a list of the design patterns present in the GOF book and framed them as the main concepts of OOP, I would imagine people would be a little bit suspect, right?
That's because it's kind of the reverse, isn't it? The main concepts of OOP may result in certain common PATTERNS of code structure and layout -- which have usually been given names like "Strategy" and "Visitor" and "Singleton" -- but those patterns are not themselves the main concepts of OOP.
Likewise, data-oriented design might lead you to convert an array-of-structures into a structure-of-arrays or avoid branching on a hot path but those patterns are not themselves DOD.
In my understanding of DOD, I'm not sure there really is much of a basis that can be explored in a general way, beyond just adapting to whatever platform you are ultimately targeting. "The data" is supreme (this, along with an information theoretic understanding of "data," should be a hint that something is rotten in the state of DODmark; EDIT: this is extremely dismissive, which is not my intention, and I think DOD makes total sense in combination with other design approaches) and so the programmer tweaks representations of the data to best fit the platform she's working with. Thus DOD would have different positions for x86 or AArch64 or GLSL. DOD seems like a map of constraints x capabilities to strategies. For example, use struct-of-arrays when you need to access sequential elements repeatedly in a von Neumann system with CPU cache. A GPU-based solution might be entirely different, involving weird zipping and index mapping or something. The DOD approach (if it can be called that) seems undecidable when considering how an FPGA might be configured to solve this (or any) problem.
Maybe I'm way off-base, and if so, please correct me. Everything I've seen with DOD (from its inception years ago) seems in line with it, though.
I think for low level high throughout systems that makes sense, but then how do you go up a level when networking gets involved, dealing with back pressure, load balancing, and lots of other things, etc.
Dod taken to its extremes, is to give up on alot of other concepts like avoiding data duplicates. Usually you build a oop prototype to identify the hot loop and then build datastructures optimized for this process. Imagine it like a chemical plant that no longer receives one type per train or wagon, but instead processor batch sized pre mixed ingredients on wagons, that might even be reused as storage and staging for the refinery traveling overhead. A classic oop objects attributes might be distributed over several of such data wagons and the problem here is synchronization and updating. So what if your Algo processed it fast but is bow stuck waiting for another to sync its data back to arrays of optimal accessstructs.
Likewise the focus on ECS architecture as being the one true DoD pattern when it’s not necessarily data oriented at all and the unfounded assumption it’s used everywhere in game development when it isn’t . And somehow the idea that DoD is a replacement for OOP when it’s really an orthogonal concern.
DoD is about recognising data use patterns in your software and organising your software architecture around them in a way that suits the constraints of modern hardware. That’s all.
> And somehow the idea that DoD is a replacement for OOP when it’s really an orthogonal concern.
I’m sure this is true for some definition of OOP, but I’ve seen too much OOP code that insists on hanging every method off of the most plain-old-data of classes. A Book class has to have a buyFromAmazon() method and consequently each instance has a private reference to an Amazon SDK client, and if you want to buy 10K books, you iterate over a list and invoke this method 10K times.
Of course, some will argue that this is bad OOP and true OOP doesn’t look like this, and anyway you can write bad code in any paradigm! Of course, OOP is uniquely plagued with bad code (much more so than other paradigms) and this pretty transparent no-True-Scotsman argument is just moving semantic goalposts (as such arguments do).
Whilst it’s impressive you’ve managed to hold an argument with yourself I think it’s more the case that anything as broad as a programming paradigm will naturally hold multiple approaches. Is immutability a defining feature of FP? Let the battles commence.
Likewise critics might focus heavily on inheritance or some other feature of OOP that is easy to critique.
And whilst I’m not particularly interested in defending OOP I think describing it as uniquely plagued with bad code is not something anyone should just take axiomatically.
> Whilst it’s impressive you’ve managed to hold an argument with yourself
I like to address the boringly predictable responses up front so we can avoid rehashing the same silly arguments over and over. I apologize if that spoiled your fun.
> Is immutability a defining feature of FP? Let the battles commence.
I think most would agree that immutability is more prevalent in FP even if struct immutability isn’t required. Moreover, everyone would agree that first class functions and functional composition are key characteristics. Contrast that with OOP where you have some OOP enthusiasts arguing that inheritance is a defining feature and others who argue it isn’t. Some argue that the “kingdom of nouns”, banana-gorilla-jungle design is inherently OOP and others argue it’s “bad programming and not true OOP”. Others argue that message passing is required, but many others argue to the contrary. While other programming paradigms have fuzzy edges, OOP has no discernible shape at all.
> And whilst I’m not particularly interested in defending OOP I think describing it as uniquely plagued with bad code is not something anyone should just take axiomatically.
Why are there no equivalent criticisms of FP or DO? We might find FP codebases that are overly abstract or a bit slow, but we don’t tend to find (m)any that are designed such that a Book object holds a reference to an Amazon SDK client or a banana with a reference to the gorilla holding it with a reference to the entire jungle. There aren’t prevalent guidance to write code like that in other communities like there is (or perhaps “was”) in OOP circles. Similarly, there aren’t enterprise-y abstract factory beans or anything like that in the FP world.
Mind you, I’m not dumping on OOP—indeed I couldn’t if I wanted to because it has no agreed upon definition, per its proponents.
I think the problem with OOP is that the paradigm itself doesn't offer much. It's a very generic paradigm with a massive sandbox. It needs to be more opinionated.
This is only "bad OOP" if you need to buy 10K books at once and it is actually too slow and there is a significant amount of fat to trim (if you need to call a web service once for every book, incoherent memory access is likely to be negligible).
Otherwise it's obvious, cheap to write, easier to get right than more complicated approaches, and good enough. True OOP is OOP that meets goals.
Other approaches aren’t more complicated, and this is worse even if you never need to send books in batch because it tightly couples “book” to the Amazon AWS SDK client. Anything that interacts with books now has to take a dependency on the Amazon SDK. A much simpler, better design would be to just call client.buyBook(book) or client.buyBooks(books).
But more importantly, my point is that you have your definition of whether this is OOP or not but lots of OOP proponents will say that this is not true OOP.
Actually, the Amazon client could be a component that is managed by a sane dependency injection system (e.g. a factory of "books that can be bought on Amazon") and is used by a book to implement the abstract book operation "buy a copy of me".
This would be basically equivalent to a "book buyer" object with a "try to buy this book" operation, with small advantages and disadvantages (either the books or the bookstores are privileged as the main entity in the many to many "can be bought at" relationship).
Better, but the book still needs a dependency on your "book buyer" (let's call it "Retailer" for sanity) interface even though there are likely lots of things to do with a book besides buy it, and those applications shouldn't have to care about the details of book buying. For every verb, the book field needs at least one new field to support that verb (e.g., the `Book.retailer` field to support the `buyBook()` method), even though each thing you might want to do with a book involves at most a few of those fields.
Moreover, inevitably someone downstream from the author of the Book class will have a use case for dealing with books that the author hasn't accounted for, so they have to extend the book class for their own use case, tacking ever more fields onto that book even though their use case only cares about a few of the fields.
Additionally, some of the things you might want to do with a book might also involve some other plain-old-data-structure--how do you determine which plain-old-data should host the method? Why is it `Book.doThingWithCar(Car c);` and not `Car.doThingWithBook(Book b);` or simply a static method `doThing(Car c, Book b);`?
Further still, why create a Book.buy() method that's just going to delegate to `Retailer.buyBook(this)` anyway? If the advantage is that you don't have to explicitly pass the `retailer` around, that's fine enough but there are ways to do that without baking retailer details into every book instance (e.g., a closure: `buyBook = function() { retailer.buyBook(book) }` or if you're a glutton for punishment, you can create a class weds a book and a retailer together:
class BuyBookOperation {
private Retailer _retailer;
private Book _book;
public BuyBookOperation(Retailer retailer, Book book) {
this._retailer = retailer;
this._book = book;
}
public do() { this._retailer.buyBook(this._book); }
}
Further, you might want to buy a book from many retailers--why should you have to do `book.setRetailer(amazonClient); book.buyBook(); book.setRetailer(barnesAndNobleClient); book.buyBook();`? Why not simply `amazonClient.buyBook(book); barnesAndNobleClient.buyBook(book);`? Even if you abstract away the mutation by creating a `Book.buyFromRetailer(Retailer r)` method that sets the retailer and then calls Book.buy(), you now have a potential race condition in parallel code and you still have no advantage over retailer.buyBook(book).
Lastly, if at some point you do need to buy books in batch, how do you support that? Does each book class also need a reference to every List<Book> that references it so it can do book.buyInBatch()? Do you make a book.buyInBatch(List<Book> otherBooks) method? Do you just eat the performance hit and make individual calls to book.buy() even though the Retailer interface has a buyManyBooks(List<Book> books) method that could buy all books in a single HTTP request? What advantages do these approaches have over `retailer.buyManyBooks(books)`?
So this "plain old data needs to depend on everything that might be necessary for any activity involving a book" pattern has no obvious benefits, but it creates a lot of unnecessary coupling, creates a lot of awkward interface decisions when there are other objects involved in an action (including efficiently dealing with collections of your plain old data structure), and it probably pushes you into mutation unnecessarily which makes it a lot more difficult to parallelize your code.
This pattern seems to be a pretty transparent anti-pattern to me, and if it's "inherently OOP" then OOP is problematic.
> the Rust game dev community is overly fixated on ECS.
Not just Rust. The game dev community everywhere is infatuated with ECS.
It's basically cargo culting. There is a large base of amateur or indie game developers who want to feel like they are doing game development the "right" way. One big aspect of that is performance. ECS has a reputation for efficiency (which is true, when used well in a context where your performance problems are related to caching), so you see a lot of game devs slavishly applying it to their code in hopes that the "go as fast as a AAA game" Gods will land on their runway and deliver the goods.
Every time I see a new ECS framework in JavaScript, I die a little on the inside.
The problem is that software design in general is an incredibly messy field that's still basically in its infancy. Developers want simple solutions to complex design problems, or at least they want some decent architectural guidelines so they can avoid constantly reinventing the wheel, badly. Remember when MVC was all the rage?
ECS is good though, it's a perfectly solid answer to a lot of thorny design questions. Where problems frequently arise is when you try to jam absolutely everything in your game into the ECS structure. In practice you're probably going to have a lot of data which lives outside the system and is not attached to any entity.
Most of what I see about ECS is how much easier it is to have dynamic behaviors without inheritance, and it is, so I don’t see why it would be bad for newcomers to use it or to have an ecs lib written in js.
> how much easier it is to have dynamic behaviors without inheritance
I think you're getting at the idea that instead of having objects differ in their behavior by overriding methods, you have them differ by having fields bound to objects that implement different behavior.
Assuming I understand you right, that's an excellent insight, but it's just the classic principle:
Favor object composition over class inheritance.
There's nothing new in that idea or anything special to ECS. Do you know how I know? Because that sentence here is directly quoted from page 20 of "Design Patterns", which ECS and DoD are often claimed to be in direct opposition to. :)
> I think you're getting at the idea that instead of having objects differ in their behavior by overriding methods, you have them differ by having fields bound to objects that implement different behavior.
I guess I'm more getting at the idea of changing the game design at any point by adding or removing components, a way to make it easier for devs to cope with changing requirements (and they are always changing ofc), but you are right about favoring composition over inheritance, that by itself is pretty good.
I can't really talk about "things that are often claimed" and from the way you talk about this it seems like you have come across different opinions from mine on what ECS is or its value. Sad to see such a useful pattern get "corrupted", but I suppose that is inevitable.
Not only that, it is incredible how it gets trumped as a new idea, when stuff like COM, Objective-C protocols and plenty of other component based architectures were already a subject in OOP related papers during the late 90's.
It's a totally different composition pattern though. It uses more trait-style multiple inheritance, unlike the typical "has-a" composition typically used in object-oriented languages. Additionally, it intentionally breaks encapsulation, which is a key tenet of object-oriented design.
I wouldn't dismiss the JS ECS frameworks without measurement.
Polymorphism has costs, and while dynamic languages work hard to remove them, they still work best when they're able to monomorphize the call site, because that enables inlining without combinatoric explosion from chained polymorphic calls.
Having a single type in your array means field accesses, method calls etc. have the potential to be monomorphized. There are performance wins to laying out your data in ways that avoid the need for polymorphism.
> I wouldn't dismiss the JS ECS frameworks without measurement.
I think the burden of proof is on the part of JS ECS frameworks to show they do have better performance by virtue of DoD and, if so, why. JS engine programmers have been optimizing object-oriented code for literally forty years, all the way back to when they were making Smalltalk VMs.
If somehow a couple of folks hacking on ECS frameworks have managed to write code that runs faster on those JS engines than the kind of code they were designed for, I'd like to see it.
> Having a single type in your array means field accesses, method calls etc. have the potential to be monomorphized.
Sure, but object-oriented code does not require any more polymorphism than DoD does. Consider:
* Iterate over an array of monomorphic components and call a method on each one.
* Iterate over an array of monomorphic entities, access a monomorphic property, and call a method on the latter.
There's an extra property access in the latter (which can easily be inlined), but no polymorphic dispatch. In practice, yes, it is possible to reorganize your JavaScript code in ways that play nicer with inline and possibly even code caching. But I have never seen any evidence that JS ECS frameworks actually do that. Instead, the few I've poked around in seem like typical slow imperative dynamically-typed JS.
If someone is going to take a pattern that was invented specifically for a language like C++ that gives you precise control over memory layout and then apply it to a language that not doesn't give you that control but often uses hash tables to store an object's state, I think the burden of proof is on the framework to show that the pattern actually applies.
That one's pretty interesting. Here you can see they are putting real effort into thinking about the performance of the underlying VM. Using typed arrays is neat.
Modern JS engines won't use a hash table for the object state, they'll use a hidden class, and member accesses will be fixed offset indirect loads guarded by a type check. Initialize your objects carefully in a deterministic order, and I'd expect you can control field order and adjacency.
I'd expect the wins from reworking your JS so that your target JS engine lays out data out better would often be larger than the wins in C++, simply because the worst case is so bad.
I'd add a third option to your pair: iterating over several arrays of properties - especially numeric properties - rather than an array of objects which each have numeric properties. That can get you substantial wins in Java, never mind JS.
> Modern JS engines won't use a hash table for the object state, they'll use a hidden class, and member accesses will be fixed offset indirect loads guarded by a type check.
The type checks themselves have significant overhead, and it's easier to fall off the shadow class fast path than you might expect.
> Initialize your objects carefully in a deterministic order, and I'd expect you can control field order and adjacency.
True, but that's equally true of non-ECS architectures. I have yet to see much evidence that the ECS JS engines I've looked at are actually taking that into account.
> ... so you see a lot of game devs slavishly applying it to their code in hopes that the "go as fast as a AAA game" Gods will land on their runway and deliver the goods.
I watched this sentence unfold with bated breath, waiting to yell "and ze sticks the landing!", only to see it end with "goods" instead of "cargo". It's frustratingly close to perfect, though perhaps to end the paragraph with "cargo", you'd have to begin with some synonym for cargo culting.
I agree that Rust is more suited to ECS than hierarchies. However, the choice is not between ECS or inheritance. The reason I say that the community is overly fixated on it is that many of the benefits attributed to ECS aren't unique to ECS.
I’d agree to a point but it really crosses the whole gamut of hobby game engine development. It’s weirdly self reinforcing even in the face of more interesting architectural choices like DOOM Eternal’s.
The way Eternal's engine is described it could simply be an ECS with a more advanced job system for farming out work because it can parse how the various objects are updated and only do reads after all writes to the objects happen.
I believe it’s only been mentioned in passing and likely will be talked about at conferences soonish. Here’s a HN thread from when it was first talked about:
There was an early talk about using a job system to run the Destiny renderer and then this one for the whole engine which is a very similar premise to the way DOOM(2016) and then DOOM Eternal evolved.
Basically it boils down to making use of Objective-C protocols or multiple inheritance, with data separated from behavior in regards to implementation.
So a 1986 concept rediscovered by game developers and cargo culted ever since, although it has its roots on the OOP they love to hate.
That's okay, now that you're familiar with the term, you'll end up noticing it a lot more (or maybe it's just that articles that happen to lend themselves to it showing up in their comments look slightly more appealing now?).
Taking this article as an example, it frames DOD as an optimization technique and explicitly states that these patterns are the main concepts of DOD.
But while these patterns are interesting and often show up in data-oriented designs, they are not themselves foundational to data-oriented design.
This is interesting to me because it seems to obviously be missing something. If the article went through a list of the design patterns present in the GOF book and framed them as the main concepts of OOP, I would imagine people would be a little bit suspect, right?
That's because it's kind of the reverse, isn't it? The main concepts of OOP may result in certain common PATTERNS of code structure and layout -- which have usually been given names like "Strategy" and "Visitor" and "Singleton" -- but those patterns are not themselves the main concepts of OOP.
Likewise, data-oriented design might lead you to convert an array-of-structures into a structure-of-arrays or avoid branching on a hot path but those patterns are not themselves DOD.