This is the true power of Rust that many are missing (like Microsoft with its TypeScript rewrite in Go): a gradual migration towards safety and the capability of being embedded in existing project.
You don't have to do the Big Rewrite™, you can simply migrate components one by one instead.
> like Microsoft with its TypeScript rewrite in Go
My understanding is that Microsoft chose Go precisely to avoid having to do a full rewrite. Of all the “modern” native/AoT compiled languages (Rust, Swift, Go, Zig) Go has the most straightforward 1:1 mapping in semantics with the original TypeScript/JavaScript, so that a tool-assisted translation of the whole codebase is feasible with bug-for-bug compatibility, and minimal support/utility code.
It would be of course _possible_ to port/translate it to any language (Including Rust) but you would essentially end up implementing a small JavaScript runtime and GC, with none or very little of the safety guarantees provided by Rust. (Rust's ownership model generally favors drastically different architectures.)
No, it was absolutely about the effort needed to rewrite the project. They couldn't afford a rewrite, only a port. They're not going to keep maintaining the Typescript version once they have transitioned to the Go version.
Yes, they distinguish between a rewrite and a port (first time I heard the distinction like that, but it intuitively makes sense).
A Go port looks roughly the same as TypeScript, same “shape”, same concepts, so they don’t need to re-architect the code, they can just “translate” TypeScript to Go, then clean up where makes sense or needed. For a good while (definitely years, probably half a decade, if you ask me) while both projects are maintained, adding fixes and features will be therefore easy. The two codebases can be expected to have almost the same output and bugs, both is good for maintainability.
With Rust, the translation wouldn’t work as Rust is significantly different. This would mean rethinking everything, the architecture would diverge, resulting in two possibly very different set of bugs. With different structure, tweaking both at the same would be very difficult.
In a similar way Rust can be very useful for the hot path in programs written in Python, Ruby, etc. You don't have to throw out and rewrite everything, but because Rust can look like C you can use it easily anywhere C FFI is supported.
IIUC word tearing in Java cannot cause arbitrary memory corruption. Any read is guaranteed to see either the old value, or the new value, or half of each. Since these are just numbers we're talking about, not pointers, and the language doesn't otherwise allow memory-unsafe things like pointer arithmetic, a data race resulting in word tearing in Java can at worst result in a runtime crash or a logic bug, not arbitrary memory corruption.
By contrast, in Go, all it takes to cause full-blown undefined behavior, in the same sense as C or C++ or Rust, is accessing an interface/map/slice value in a racy way, not having the race detector on, and being sufficiently unlucky.
I agree that in practice this doesn't stop people from calling Go a memory-safe language. I think it's reasonable to argue that this statement is just wrong. That said, I think the Carbon people once said something along the lines of, in their experience, it's rare in practice for memory corruption bugs in C++ to be caused solely by data races. So a language that's memory-safe in single-threaded contexts, but where concurrency is not memory-safe, is "good enough". Maybe it's the same in Go.
I am still suspicious of this, though. Maybe it's just that concurrent operations are rarer in general because they're so famously hard to get right, and this makes it less likely that you'll have one that's exactly the right kind of wrong to cause memory corruption. Certainly it seems at odds with the notion that you want the exceptions to memory safety to be auditable.
Yes, you are correct that Java's guarantee is stronger.
> By contrast, in Go, all it takes to cause full-blown undefined behavior, in the same sense as C or C++ or Rust
It's a little more tricky than that. UB in C/C++/Rust is something that the compiler can use to transform the code. This can lead to other issues. Go's compiler will not do those kinds of things. So on one hand, yes, it can lead to arbitrary Bad Things, but on the other hand, it is different.
> Maybe it's the same in Go.
I was having a discussion about this topic last week, regarding Uber's paper from a few years back trying to analyze the prevalence of these issues: https://news.ycombinator.com/item?id=43336293 You'll notice the person replying to me has pointed out some additional mitigations that have happened since.
There is a real practical problem with the dichotomy in Go unfortunately, where on the one hand a single threaded program is safe, but on the other hand no go programs are single threaded. More mitigations are coming and synctest for example is quite welcome and is already helping where we've tested it, but this stuff just feels like the same old whack-a-mole as e.g. fuzzing for other solutions. gvisors checklocks for example isn't tenable for us without hugely more investment as there are tons of cases it just can't handle (like inline funcs, which we have more than we ought to). You're right that it's different, though - data races are often much harder to discover, arrange and leverage.
We've witnessed memory corruption events (from plain go code), and it's raised real concerns even among our strongest advocates. Our most recent case was actually a false-positive, in that there was a real race on a map, but it so happened it would never have reached a data race - still it tore down a process in production, and this is good and bad on multiple fronts. It doesn't make the solution invalid, and no one should be throwing internet punches over it, but it's absolutely reasonable to be concerned about memory corruption cases in complex go programs. In general my advice for Go usage is to write small programs with low concurrency and small heaps, this manages associated risks well, both in terms of authorship and exposure when events come up. Small container sized things are probably fine, essentially, but many tens of gb of user data in heaps with hundreds of constantly changed kloc is a place to be genuinely concerned.
Reflecting back on the starting context, and this advice, they align. The compiler has no high value user data to corrupt, if it suffers memory corruption the bad cases have to be pretty lucky not to end up in sufficiently total failure as to cause true end user problems. The runtime is short lived and compartmentalized, and it's not in a highly adversarial environment - this is a fine fitting use case. In an attacker heavy situation like the font system though, essentially I'm almost ready to drop maybe one layer of sandboxing once it's implemented with stronger guarantees and had some appropriate testing. I'm not sure I'd be ready to say that with less safety guarantees.
Java only has tearing for double and long, which are POD types and so are irrelevant to memory safety. Go has tearing for pointer-like objects (interfaces, slices), which causes the memory safety problems.
Data races on those pointer-like objects are undefined behavior in Go, whereas there isn't something comparable in Java. The reason why Go is considered a memory safe language is actually pretty mundane: it's simply that the data race issues are pretty hard to exploit in practice for a variety of reasons.
Memory model strength has nothing to do with tearing. Total store order and total load order does not matter in any possible way.
There's no tearing, because all memory operations are wider than 64-bits. Unless someone purposefully performs two 32-bit operations on a single 64-bit value.
It's been quite a while since I worked with golang details but I seem to remember writes to the same map from multiple goroutines being UB if GOMAXPROCS>1? Did they eventually solve that, or is that excluded from the definition of memory-safe language because it's a different type of safety?
This is my understanding (both that that can occur, and that it does not count as memory safe).
I called out the types I did in my post specifically because they're the types where data races can result in arbitrary memory corruption (in the actual implementation, the spec is arguably not even that strict). Per https://go.dev/ref/mem#restrictions (the same linke as in Steve's reply to me)
You don't have to do the Big Rewrite™, you can simply migrate components one by one instead.