I get the urge to be cynical all the time, but this isn't that time. "Once you grow", they have already grown and competing with the SoTA models and still giving it all back to the community.
I just wish this smear campaign against them stops sometime soon.
Agreed but this isn't the same as an open source library; it costs A LOT of money to constantly train these models. That money has to come from somewhere, unfortunately.
Yeah. The amount of compute required is pretty high. I wonder, is there enough distributed compute available to bootstrap a truly open model through a system like seti@home or folding@home?
Distributing the training data also opens up vectors of attack. Poisoning or biasing the dataset distributed to the computer needs to be guarded against... but I don't think that's actually possible in a distributed model (in principal?). If the compute is happing off server: then trust is required (which is not {efficiently} enforceable?).
Trust is kinda a solved problem in distributed computing, The different "@Home" projects and Bitcoin handle this by requiring multiple validations of a block of work for just this reason.
How do you verify the work of training without redoing the exact same work for training? (That's the neat part: you don't)
Bitcoin is trust-solved because of how the new blocks depends on previous blocks. With training data, there is no such verification (prompts/answers pairs do not depend at all on other prompt/answer pairs) (if there was, we wouldn't need to do the work of training the data in the first place).
You can rely on multiplying the work where gross variations are ignored (as you suggest): but that will take a lot more overhead in compute, and still is susceptible to bad actors (but much more resistant).
There is no solid/good solution - afaik - for distributed training of an AI (Open assistant I think is working on open training data?), if there is: I'll sign up.
There has been some interesting work when it comes to distributed training. For example DiLoCo (https://arxiv.org/abs/2311.08105). I also know that Bittensor and nousresearch collaborated on some kind of competitive distributed model frankensteining-training thingy that seems to be going well. https://bittensor.org/bittensor-and-nous-research/
Of course it gets harder as models get larger but distributed training doesn't seem totally infeasible. For example if we were to talk about MoE transformer models, perhaps separate slices of the model can be trained in an asynchronous manner and then combined with some retraining. You can have minimal regular communication about say, mean and variance for each layer and a new loss term dependent on these statistics to keep the "expertise" for each contributor distinct.
Forward-Forward looked promising, but then Hinton got the AI-Doomer heebie-jeebies and bailed. Perhaps someone picks up the concept and runs with it - I'd love to myself but I don't have the skillz to build stuff at that depth, yet.
>> but Y-Combinator literally only exists to squeeze the most bizness out of young smart people.
YC started out with the intent to give young smart people a shot at starting a business. IMHO it has shifted significantly over the years to more what you say. We see ads now seeking a "founding engineer" for YC startups, but it used to be the founders were engineers.
>> Training these big models is very very expensive.
Which is why they are not the future. A big model that can generate a picture about anything in response to any input makes for a great website. It generates lots of press. But it is not a reasonable tool for content generation. If you want to produce content in a specific area or genre, the best results come from a model trained or modified in the area. So the big generalized AI, if you use it, would only be the framework on which you built your specialized tool. Building that specialized tool, such as something dedicated to images of a particular politician, does not require huge amounts of computation. That sort of thing can and is being done by individuals.
I am waiting for a tool trained on publicly-accessible mugshots. It wouldn't be a very big project but could yield a tool to generate very believable mugshots of politicians.
Depending on your background and circumstances, there are ways to opt out of the race to a greater/lesser degree. Moving to a cheaper city in your country, or a cheaper country altogether, is one of them. Finding a less stressful way of making less money is another.
It's just hard being reminded that there's no escape hatch - we've welded them all shut for eternity. Being reduced to choices within a system but the choice horizon never extends to the system itself and won't within my lifetime makes me feel trapped.
They have a big commission for transactions in some countries , and some people ended up with their account blocked and having no wayu to get their money out.
I am just a user, not a merchant so I am not sure what the issues would be from that side of the transactions, I personally avoid keeping too much money in PayPal just in case they somehow block my account for some bullshit reason.
Edit: I would prefer not to go off topic, unless is related with why some big companies refuse to use PayPal and others do use it. As an example I could not buy audio books from amazon so my money went to a different company that accepted PayPal.
If a company implements paypal then people who only have paypal might pay them money that they otherwise wouldn't. However, people who might have otherwise payed via another means may also now use paypal. If paypal has higher fees this could well result in less money in revenue.
Also, maintaining each payment gateway has an implementation and maintenance cost. Add it all up, and assuming that the number of potential customers like you that only have paypal is small, and it's easy to see why companies may choose not to implement it.
Who knows. Every company is different, has different customer bases, different histories, different technologies. Maybe they signed a deal with paypal where they get lower fees, maybe they integrated it in a time when it made more sense then now (e.g. before apple pay and google pay were so prevalent), maybe they outsource their payment to a processor that just supports it.
I don't know the details of any of these places, I was just giving a reason why a company may not implement a particular payment processor. It's based on a balance of factors, not just a simple "support X get more paying customers."
While I agree that Peter's suggestion is the safe thing to do, based on the following tweet from the official account, if you were in the country legally on 06/24, the proclamation doesn't seem to apply.
<nitpick>All Unicode characters map, one-to-one, to their code points. A code point being a numeric identifier. It's a grapheme that combines multiple characters to form a unit of writing.</nitpick>
<nitpick>The Unicode standard does not have a single definition for "character" because there's multiple interpretations. One reasonable interpretation is "a grapheme cluster".</nitpick>
More specifically, here's what the Unicode Consortium glossary defines for "Character":
> Character. (1) The smallest component of written language that has semantic value; refers to the abstract meaning and/or shape, rather than a specific shape (see also glyph), though in code tables some form of visual representation is essential for the reader’s understanding. (2) Synonym for abstract character. (3) The basic unit of encoding for the Unicode character encoding. (4) The English name for the ideographic written elements of Chinese origin. [See ideograph (2).]
An accent mark by itself has zero semantic meaning in a written context. It's a modifier. But you need to know what it's modifying in order to assign it any sort of meaning. We're talking about semantic meaning within the context of a written language, not technical details.
I just wish this smear campaign against them stops sometime soon.