Funny to see this come back and see my write-up linked. I did this 8 years ago and think I was the first on this particular board (although others had done similar on other boards). I still have a pile of them sitting on my desk because I accidentally kept bricking them by being ... not careful. That said, even at the time this board was already old, so I guess it's positively prehistoric at this point. I eventually stopped working on this because I thought that others were making sufficient progress. It hasn't really fully materialized yet, but between openbmc, opensil, DC-SCM and the work the oxide folks are doing, I'm still hopeful that we'll get out of server firmware hell eventually.
Out of curiosity: how "bricked" are these boards? Is there irreversible hardware damage (and, if so, how?), or has some firmware just gotten overwritten?
One of them I managed to fry the pcie root complex somehow, not sure exactly how. One I damaged the traces to the BMC SPI flash. Two others I think just have bricked firmware, but it's been years, so I don't remember for sure.
1. First authentication didn't work on my headless system, because it wants an oauth redirect to localhost - sigh.
2. Next, WebFetch isn't able to navigate github, so I had to manually dig out some references for it.
3. About 2 mins in, I just got
```
ℹ Rate limiting detected. Automatically switching from gemini-2.5-pro to gemini-2.5-flash for
faster responses for the remainder of this session.
```
in a loop with no more progress.
I understand the tool is new, so not drawing too many conclusions from this yet, but it does seem like it was rushed out a bit.
Similar. Yesterday things seemed to be going okay. It was trucking along, making code changes.
Then I hit the rate limit. - Fine, no worries, it'll be interesting to see if the quality changes.
Then it starts getting stuck and taking forever to complete anything. So I shut it down for the day.
Today, I start it back up and ask it to pickup where it left off and it starts spinning. I forget about it and come back 7.5 hours later and it' still spinning. When I kill it it said: 1 Turn, 90k input tokens, 6.5 hours of API time... WTH?
And now I'm just totally rate limited - `status: 429, statusText: 'Too Many Requests'` - every time. Also, I can't find any kind of usage data anywhere!
I got the same experience. I ran `gemini --debug` and it spit out the authentication URL at least. However I got the same `Rate limiting detected` for a simple refactoring task in a repo with 2 python files after a few minutes.
not working for me either. getting "too many requests". my own CLI tool i wrote for gemini last week works better lol.
maybe i have something configured wrong? though that really shouldn't be the case. my api key env var is correct.
What's the motivation for restricting to Pro+ if billing is via premium requests? I have a (free, via open source work) Pro subscription, which I occasionally use. I would have been interested in trying out the coding agent, but how do I know if it's worth $40 for me without trying it ;).
We started with Pro+ and Enterprise first because of the higher number of premium requests included with the monthly subscription.
Whilst we've seen great results within GitHub, we know that Copilot won't get it right every time, and a higher allowance of free usage means that a user can play around and experiment, rather than running out of credits quickly and getting discouraged.
We do expect to open this up to Pro and Business subscribers - and we're also looking at how we can extend access to open source maintainers like yourself.
It's linked from the main website if you hit the "Log In" button and there was communication to customers about this, though I had the same initial reaction, which is why I looked around for corroboration before posting this.
My understanding is that the TinyTapeout people were using efabless as a service provider and efabless was also providing some sponsorship, but that they are institutionally distinct. There's a LinkedIn post from the TinyTapeout folks that they're looking into alternatives.
That's a relief! And Tiny Tapeout has already done a beta with IHP's open-source 130nm BiCMOS SiGe PDK.
The IHP PDK is really a lot more exciting to me than the Skywater stuff because it's aimed at submillimeter analog things (450GHz fₜ, 650GHz fastest oscillator) and why would you fab a digital design in 130nm instead of just programming an FPGA?
> why would you fab a digital design in 130nm instead of just programming an FPGA?
That’s an interesting concept. So an fpga implemented on a current 7nm process is more performant (clock speed and energy use) than an asic on a 130nm process? How about 40nm process? I feel like there’s a graph of intersecting lines here.
I think perf is usually relatively close between an optimized design in a 7 nm FPGA and an optimized design in ~40 nm CMOS, but it's not 1:1. The FPGAs are usually higher-performance than 130 nm, but there are certain things that are easier in ASICs (eg analog-related stuff).
Speaking as a newbie - FPGAs can't get anywhere near the same clock speed, though, right? So the equivalence only applies if the work is parallelizable?
With the exception of the highest clock speed chips (eg Intel CPUs), clock speeds can actually be comparable. 45 nm CPUs got to 2.5 GHz, and if you tickle a 7 nm FPGA just right it can get to ~800 MHz to a GHz. Things like microcontrollers and chips that are generally less optimized than the old Intel CPUs (which were mostly drawn at the transistor level and use a speed-optimized process) are much closer in speed. A 3-stage RISC-V at 45 nm is probably also running at 400 MHz or less, and the FPGA is capable of a 3 stage RISC-V at that speed.
But yes, in general, FPGAs on certain computational tasks will need deeper pipelines or the use of parallelism. Usually, pipeline depth works. Actually, if you look at the Intel front side bus (less optimized than the core), that's about the speed you can get from a 7 nm FPGA.
The Sky130 IO pads can't go faster than 33Mhz (at least the ones in the open source PDK), and the OpenLane flow isn't yet timing driven, so anything internal isn't going to break more than 100Mhz. These aren't fast chips or fast processes, Skywater is mostly for pedagogical and niche military and research tapeouts.
A few sq mm at 40 nm is about $20k, and you can only configure it once. I think the Versal also gives you more useful gates at that size (thanks to block RAMs and hard multipliers).
The FPGA will have higher static power (running all the overheads) but probably lower dynamic power for the same design. 40 nm is old at this point for high-performance chips.
The static power might also depend on whether the FPGA is an SRAM type or a floating-gate type, I'd think. Does Lattice have any parts fabbed in relatively new processes?
That price is probably $4600 for one Xilinx Versal.
For the MPW run you would get ~100 parts. When everything is said and done, and you pay for packaging etc., on a MPW run you'll likely pay something like $50K. So ~$500ea
The eFabless price of $10K for a full run, including a packaged part, was an unparalleled deal.
Radiation tolerance is one case. For the price of a tiny tapeout run you could count on one hand how many qualified radiation tolerant ICs you could buy. There's some sauce involved with process choices for radiation tolerance, but one of critical things to do is use large features.
Inaccuracies with respect to the silicon (active and passove parsitics, mechanical stress effects, temperature dependencies etc) mainly, user friendliness and proper documentation as secondary .
If you need some analog features, that's conventionally called a "mixed-signal design", not a "digital design". I wasn't talking about mixed-signal designs, for which it's obvious that an FPGA is unlikely to work.
You should really look into summaries on how deep sub-micron adds more problems as processes shrink. It's crazy that 28nm and under even work at all. They also break faster in more ways than larger, mature nodes.
Far as 130nm, I'll give you a few reasons I'd use one over a 7nm FPGA. This is a non-HW guy saying what he's heard from pro's at different times. HW people, feel free to correct me about whatever I get wrong.
1. Unit prices. If you can take the upfront cost (NRE), the per unit price will be much lower than FPGA's. You might charge plenty per unit depending on the market. This can be a source of profit.
2. Older, larger nodes are said to be better for analog. Lots of designs are mixed-signal to use analog for it's lower power, extra performance, or how it doesn't blink (no rise/fall with clock).
3. ASIC's can't be reprogrammed like FPGA's. The custom design might be more secure like Sandia Secure Processor (Score) or CHERI RISC-V. FPGA's can only do one of these except for antifuse FPGA's.
4. Larger nodes are easier to visually inspect for backdoor with cheaper, teardown hardware. Who knows what's in the FPGA's.
5. Larger nodes are easier to synthesize, P&R, and auto-inspect (eg Calibre). That means open-source tools have a better chance of working or even being developed.
6. If not too power hungry (or power is cheap), some applications can let you outperform 7nm parts with parallel use of 130nm parts which are much cheaper or highly-optimized. An example what media wanting to do distributed, massively-parallel design for doing NN training maybe with 8-bitters and on-board, analog accelerators. My inspiration, aside from old MPP clusters (eg Thinking Machines), was a wafer-scale, analog NN done before Cerebras.
7. Improved reliability in general. In trusted checkers or fault-tolerant configuration, I feel like the 130nm parts are less likely to have a double failure or fail before the 7nm nodes.
8. If there's a business case, saying you built your own hardware is cool. It might even attract talent who benefit the company in other ways.
That's off the top of my head. Again, I just read a lot of stuff on ASIC's.
On a side note, you might find eASIC's Nextreme's interesting. They're Structured ASIC's that work like FPGA's in that design gets put on something with pre-made blocks to save money. Except, instead of software programmed, some via or metal layers get customized for the routing. While that reduces NRE cost, doing the routing in hardware supposedly reduces unit prices and energy maybe with a performance boost. They used to sample chips out quickly and relatively cheaply. Also, I think Triad Semiconductor had S-ASIC's with analog stuff.
eASIC Nextreme sounds like a good ol' fashioned ULA (uncommitted logic array), the sort of thing that's at least as old as the Sinclair ZX81 (where it drove the per-unit cost through the floor).
I hadn't heard of that. Looking it up, it's a type of gate array which I believe inspired both S-ASIC's and devices like FPGA's. Here's an intro to each for those following along:
That put having chips made into the realm of possibilities for even a small business. Other costs might prevent that but I could see more stuff opening up. I also envisioned hard blocks done on those nodes for common components so the S-ASIC was used for custom logic (eg differentiators).
Thanks! Yeah, for analog the case is obvious, both because there's no such thing as an analog FPGA and because smaller feature size comes with big drawbacks for analog; that's why I said, "why would you fab a digital design in 130nm". The others I'm less sure about, but they do sound plausible.
Yeah, there have been a dozen different attempts to make "an FPGA, but analog". This is one of them. They all failed. You'll note the page hasn't been updated since 02006: https://web.archive.org/web/20060715013941/http://www.anadig.... Analog circuits aren't fungible the way digital circuits are.
I'm not saying it's not a worthwhile research direction, just that it hasn't borne commercially viable fruit so far, despite decades of attempts.
Basically yes, but you have to generate the GDS-II from your Verilog yourself, you have to pay them several thousand dollars, the turnaround time is nearly a year, and your first and maybe second and third tapeout will probably have bugs that keep it from working at all.
There are open tool chains that will compile your design using the cells defined in the outreach programs fab specific standards. However, it will not necessarily function like your simulated hardware design.
Getting the hardware cell simulation working is not trivial, and Synopsys charges more per seat than most startups spend on labor in a year.
I really love Oxide to an unhealthy amount (it's become a bit of a meme among my colleagues), but sometimes I do wonder whether they went about their go-to-market the right way. They really tried to do everything at once - custom servers, custom router, custom rack, everything. Their accomplishments are technologically impressive, but, as somebody who is in a position to make purchasing decisions, not economically attractive. They're 3x more expensive than our existing hardware, two generations behind (I'm aware they're on track for a refresh) and don't have any GPUs. E.g. what I would have loved to see is just an after-market BMC/NIC/firmware solution using their stack. Plug it into a cheap Gigabyte system (their BMC is pluggable and NIC is OCP) and just have the control plane manage it as a whole box. I'd have easily paid serveral thousand $ per server just for that. All the rack scale integration, virtualization, migration, network storage, etc stuff is cool, but not everyone needs it. Get your foot in the door at customers, build up some volume for better deals with AMD, and then start building the custom rack stuff ... Of course it's easy to be a critic from the side lines. As I said, I do really love what the Oxide folks are doing, I just really hope it'll become possible for me to buy their gear at some point.
First, thanks for the love -- it's deeply appreciated! Our go-to-market is not an accident: we spent a ton of time (too much time?) looking at how every company had endeavored (and failed) in this space, and then considering a bunch of other options besides. Plugging into a "cheap Gigabyte" system wouldn't actually allow us to build what we've built, and we know this viscerally: before we had our system built, we had to have hardware to build our software on -- which was... a bunch of cheap Gigabyte systems. We had the special pain of relearning all of the reasons why we took the approach we've taken: these systems are a non-starter with respect to foundation.
You may very well not need the system that we have built, but lots of people do -- and the price point versus the alternatives (public cloud or on-prem commodity HW + pretty price SW) has proven to be pretty compelling. I don't know if we'll ever have a product that hits your price point (which sounds like... the cost of Gigabyte plus a few thousand bucks?), but at least the software is all open source!
Please forgive my tergiversation. I fully trust that you know your path and I know how annoying it is to be why-dont-they-just'd. As I said, I'm rooting for you.
There's two dictionary definitions of tergiversate. One is the one you quoted, the other is one of desertion. Both meanings of the word are pejorative in the sense that the word comes with a connotation of betrayal of a cause. What I wanted to express was an acknowledgement that I understood the feeling that you get when someone who's clearly a fan of your work nevertheless does not provide a clear endorsement. It's easy emotionally to dismiss people who "just don't get it". But when someone does get it but chooses to equivocate, that can feel like an emotional betrayal. So I was looking for a word that covered that with the right connotation. I originally used apostasy, but it didn't feel quite right, because I wasn't really renouncing, more failing to fully endorse, so tergiversation it was. Of course having to write an entire paragraph to explain your word choice kind of defeats the purpose of choosing a single well fitting word over just writing a sentence of simple words that explains what you mean. But hey, I write enough technical writing, documentation, reports, grants, etc. all day where clarity is paramount that I feel like I get to have a little vocabulary treat in my personal writing ;).
We are definitely very much building a business! We have the iconoclastic belief that you can build a business by having a terrific team building a great product that customers love. And we're getting there![0]
Oxide are doing great work. Hoping they can probe the market a bit more for us out on the sidelines preparing to drop in and compete with some similar tech.
Id also wish I could get to play around with a cheaper version of their tech, but they probably havw enough customers that really want a large-scale solution that is completely customizable
I don't really know what kind of rebuttal you're looking for, but I will link my HN comments from when this was first posted for some thoughts: https://news.ycombinator.com/item?id=31396861#31398796. As I said, in the linked post, I'm quite skeptical of the business of trying to assess relative buginess of programming in different systems, because that has strong dependencies on what you consider core vs packages and what exactly you're trying to do.
However, bugs in general suck and we've been thinking a fair bit about what additional tooling the language could provide to help people avoid the classes of bugs that Yuri encountered in the post.
The biggest class of problems in the blog post, is that it's pretty clear that `@inbounds` (and I will extend this to `@assume_effects`, even though that wasn't around when Yuri wrote his post) is problematic, because it's too hard (arguably impossible) to write correctly. My proposal for what to do instead is at https://github.com/JuliaLang/julia/pull/50641.
Another common theme is that while Julia is great at composition, it's not clear what's expected to work and what isn't, because the interfaces are informal and not checked. This is a hard design problem, because it's quite close to the reasons why Julia works well. My current thoughts on that are here: https://github.com/Keno/InterfaceSpecs.jl but there's other proposals also.
> Another common theme is that while Julia is great at composition, it's not clear what's expected to work and what isn't, because the interfaces are informal and not checked.
This is THE huge issue when combined with the other variables:
- you often have hundreds of depedencies that interlock
- you often have dependencies that work with multiple packages, and they will often pull them all into your tree (there are sometimes "bridge packages" written, but that is O(n^2) in the number of packages)
- many maintainers do not care about good versioning practices, because they are not enforced, and it is not their problem when they break a bajillion other packages
- many people writing Julia code are not software developers. this is great! but they usually don't look at what their testing coverage looks like. often the CI always fails on a package and it's ignored.
Would Julia only allow multiple dispatch as a link between those packages, the situation would look a little better. But often the packages also talk through simple accesses of values like `X.someproperty`. I've seen situations where maintainers would add and remove these properties and break their own other libraries. Better "enforcement" of these types of things - however that would look like - would be a huge improvement for the time sink that is maintaining a Julia package.
> But often the packages also talk through simple accesses of values like `X.someproperty`. I've seen situations where maintainers would add and remove these properties and break their own other libraries. Better "enforcement" of these types of things - however that would look like - would be a huge improvement for the time sink that is maintaining a Julia package.
I think this is a cultural issue to a large degree. I think even if there were better enforcement, it's just a question of coding of discipline to then actually not break something. If it were easier to anticipate what sort of breakage a change could entail (say, by making clear & documented _by the core language_ what is considered a breaking change, and then actually not even doing "technically breaking" changes and using deprecations instead), this COULD change.
That being said, that requires a very different development workflow than what is currently practiced in the main repos of the language, so there's quite a bit of an uphill battle to be had (though such issues happen there all the time, so solving that would actually help here the most, ironically).
That sounds like Julia needs some kind of test suite, that lives alongside the major libraries used in Julia, and verifies that things that interact do so in a reasonable manner e.g. things that implement addition obey the rules of addition, etc, etc.
With a little reflection and some automation, such a tool could be taught to test new Julia libraries and catch problems early.
i.e. what I am talking about is effectively informally encoding protocols/interfaces and suchlike in a test-suite, rather than seeking to make it a part of the compiler/language.
Not as sexy, no doubt, but flexible and with a short time to get feedback.
This is precisely what we did with the SciML ecosystem. You can see a blog post from a year back: https://sciml.ai/news/2022/10/08/error_messages/. There's a lot of high level interface checking that goes on now through a trait system to ensure that the pieces are solvable before hitting code, and throwing high level error messages back based on those.
Over the last year we've been growing the domain of these as well, and have seen a decrease in bug reports come from it.
Congratulations on the launch! I think it's always a great thing to have people who know what they're doing put a new design out there. Raises the bar for everyone. For example, I think the Rust folks' work on error messages has really raised the bar on what is expected of systems is that regard. Sometimes people working on older systems feel a bit uncomfortable they see the bar being raised on them (it's weird to me to think of Julia as on "older" system, but I guess being around for more than a decade counts), but I actually prefer to think about it in the opposite way. There are lots of forces in systems design that push production systems towards conservatism. "We now have a million users, is it really worth spending the time/effort/risk on this new experimental parser/optimizer/feature, etc?", but if the rest of the world raises the bar, it's a great incentive to keep up and make all our systems better. So I very sincerely wish you the best of luck here and that in the areas where there might end up being overlap between Julia and Mojo people will start complaining to us that we need to be better, because we might just take them up on it ;).
Yes, it's useful when you're ssh'd into some production system somewhere and just want to get a quick sense of some data in memory instead of going through the steps of installing a full graphics stack (which fortunately is pretty much a one-liner in Julia also, but can take a few mins).
