A personal guideline for a lot of stuff is that a function may be too long when people add comments to mark what sections of it do. (ofc not really a hard rule). I just think it's easier to see "oh this is calling the load_some_stuff function, which I can easily see returns some data from a file." Rather than <100 lines of stuff, inlined in a big function, that you have to scan through to realize it loads some stuff and/or find the comment saying it loads some stuff>. That is to say, descriptive functions names are easier to read than large chunks of code!

smaller functions are also usually easier to test :shrug:

I kind of dislike the benchmarkification of AI for science stuff tbh. I've encountered a LOT of issues with benchmark datasets that just aren't good... In a lot of cases they are fine and necessary, but IMO, the standard for legit "success" in a lot of ML for science applications should basically be "can this model be used to make real scientific or engineering insights, that would have been very difficult and/or impossible without the proposed idea."

Even if this is a super high bar, I think more papers in ML for science should strive to be truly interdisciplinary and include an actual science advancement... Not just "we modify X and get some improvement on a benchmark dataset that may or may not be representative of the problems scientists could actually encounter." The ultimate goal of "ml for science" is science, not really to improve ML methods imo

This may not be the actual reason in this case, but I think it's good to be aware of: A non-zero chunk of "ai for science" research done at tech companies is basically done for marketing. Even in cases where it's not directly beneficial for the companies products or is unlikely to really lead to anything substantial, it is still good for "prestige"

I'm not an expert on this, so take this with a grain of salt. Chaotic PDEs are extremely sensitive to initial conditions. This essentially makes it so that any numerical solution will (quickly) diverge from the true solution over time. (Just due to floating point error, discretization error, etc.) This is why for a lot of turbulent navier-stokes stuff, people don't necessarily care about the specific phenomena that occur, but look at statistical properties.

I think one of the reasons it is important to preserve conservation laws is that, at the very least, you can be confident that your solution satisfies whatever physical laws your PDE relies on, even if it's almost certainly not the "actual" solution to the PDE. You actually can ensure that a numerical solver will approximately satisfy conservation laws. Then at the very least, even if your solution diverges from the "actual" PDEs solution, you can have some confidence that it's still a useful exploration of possible states. If conservation laws are not preserved AND your solution diverges from the "actual" PDE solution, then you probably cannot be confident about the model's utility.

Actually I just happened to see this: https://www.stochasticlifestyle.com/how-chaotic-is-chaos-how.... It's basically explaining the same thing, but much better than me :-)

it feels like Nvidia has 30 "tile-based DSLs with python-like syntax for ML kernels" that are in the works lol. I think they are very worried about open source and portable alternatives to cuda.

Not at all, they are the ones pushing for vendor agnostic Tensorcore extensions in Vulkan, which would solve some part of the portability issue: https://github.com/jeffbolznv/vk_cooperative_matrix_perf

I think very few of these "replace numerical solver with ML model" papers do anything to verify invariants are satisfied (they often are not well preserved). They basically all just check that the model approximately reproduces some dynamics on a test data of PDEs, that's often sampled from the same distribution as the training dataset...

100% of the time I get a call, it's spam, a doctor, or my mom.

I don't even get calls from my doctor anymore, everything is done online/through a portal, I just get an email with a link to my doctor's "secure portal," enter my DoB, and all our "conversations" are right there, neatly organised by topic.

If anything, if I'm getting calls from anybody in my family (or if I'm calling them), it's just assumed that it's a life-or-death problem that *genuinely* needs to be dealt with right now. Very few things in this world are so urgent that they need to be addressed right now.

I've decided 100% of papers saying their modification of a neural network is interpretable are exaggerating.

I have heard that their internal review processes for papers have started telling people to not say stuff like "XYZ may be useful for climate research" or "this is an alternative energy source that's environmentally friendly." Like they are literally discouraged from talking about climate stuff at all lol.

Wouldn't surprise me. Getting rid of the other research programs won't be great for the labs though. The weapons research has a bunch of weird incentives because of the geopolitical context it exists in. The goal usually isn't to operationalize research, it's to have credible evidence of a functioning nuclear program, maintain the arsenal, and act as a jobs program for nuclear physics. The other programs act as a way to operationalize things in socially acceptable ways. If you get rid of them, I suspect the labs aren't going to be better-off for it even with more funding.

I think they're misusing "forward propagation" and "backward propagation" to be basically mean "post training inference" and "training". they seem to be assuming n iterations of the backward pass, which is why it's larger...

n iterations would be a constant factor, which is omitted from asymptotic complexity