Rednote has been shown as the top free app (per Apple’s own App Store in my device at least) for going on a week, so the magnitude may be larger than you imply.
Also, having tried it myself, the algorithm works much like TikTok whereby it learns to show English speakers English content pretty quickly.
Also the general consensus among people who have used IG and TikTok (I personally don’t use IG) seems to be that the former does not at all substitute for the latter, particularly in terms of the subjective “authentic” feel of the content (IG often said to be lacking the community feel of TikTok).
I explicitly stated in a different comment that Rednote will not replace TikTok. I don’t think anyone seriously believes that. It’s subject to the same ban after all.
The interesting aspect here is rather the magnitude of dissatisfaction that a large percentage of users feel towards the other mainstream US social content platforms.
I don't think that's going to happen. The party official seems to be positive about the event overall based on their press release recently. IMO it's going to the opposite direction, where they try to get more foreign users on the platform and have them stay there. If I were a CCP official, I would love to have more soft power by having everyone on a Chinese platform.
P and B frames are compressed versions of a reference image. Frames resulting from DLSS frame generation are predictions of what a reference image might look like even though one does not actually exist.
But MPEG is lossy compression which means they are kind of a just a guess. That is why MPEG uses motion vectors.
"MPEG uses motion vectors to efficiently compress video data by identifying and describing the movement of objects between frames, allowing the encoder to predict pixel values in the current frame based on information from previous frames, significantly reducing the amount of data needed to represent the video sequence"
There's a real difference between a lossy approximation as done by video compression, and the "just a guess" done by DLSS frame generation. Video encoders have the real frame to use as a target; when trying to minimize the artifacts introduced by compressing with reference to other frames and using motion vectors, the encoder is capable of assessing its own accuracy. DLSS fundamentally has less information when generating new frames, and that's why it introduces much worse motion artifacts.
it would be VERY interesting to have actual quantitative data on how many possible I video frames map to a specific P or B frame vs how many possible raster frames map to a given predicted DLSS frame. The lower this ration the more "accurate" the prediction is.
Compression and prediction are the same. Decompressing a lossy format is guessing how the original image might have looked like. The difference between fake frames and P and B frames is that the difference between prediction of fake frame and real frame is dependant on the user input.
... now I wonder ... Do DLSS models take mouse movements and keypresses into account?
Speaking of Bitcoin specifically, if this is your view then you should take some time to understand the severe societal and economic problems that come with persistent deflation. And more generally, determining the optimal level of the money supply to match the needs of a dynamic economy is not trivial — there is no simple formula involving straightforwardly measurable variables that I am aware of.
Bitcoin is not a deflationary asset yet. It is a disinflationary asset until around year 2140 (with the rate decreasing every ~4 years and trending towards 0, at which point it will be deflationary).
For now its supply still inflates by ~1.7% a year.
Neither you, nor I will live to see that moment when the last bitcoin will be mined (as long as miners keep mining in our lifetime).
Bitcoin exists as an option to a lot of very inflationary assets (currencies which not that long ago almost reached 2 digit inflation rates) and having different kind of assets should be welcomed in my opinion, as it seems like a good way to diversify the risk you can't "straightforwardly measure".
I like to hold at least one asset for which I can almost guarantee a supply/emission won't change... better yet I can trustlessly verify it. If these propositions are not attractive to you or others, they don't have to acquire this asset either, it is completely opt-in, unlike the fiat money which your government forces you to acquire to pay their taxes.
> And more generally, determining the optimal level of the money supply to match the needs of a dynamic economy is not trivial
No, we don't need some grand wizards in an ivory tower to finely tune the amount of money in the economy, that idea is absurd. The economy has worked fine for thousands of years before they started doing that. In fact, since they started doing it we've seen ever growing boom and bust cycles.
Bitcoin and BFT consensus primarily solve the principal-agent problem, and give us a better means to experiment.
Personally, I don't think a deflationary global reserve currency is the precise answer, but I do believe that these experiments in monetary governance (like Ethereum's burn and mint model) will allow us to move closer to it.
Bitcoin has no governance mechanism for managing the money supply. If it was adopted as money, governments would have to implement some mechanism to manage the bitcoin supply, just like they did under the gold standard. Therefore it doesn't solve any principal-agent problem.
LLMs/transformers are a breakthrough on the level of convolutional neural networks — significant, and one that opens up lots of new interesting applications as well as invites lots more R&D — but like the neural net it’s not gonna fundamentally transform society or industry.
(And as far as I can tell this current craze is entirely predicated on LLMs/transformers.)
Autonomous cars alone will be transformative. And LLMs are not in their end-state, we went from GPT-1 to 4 in 5 years over that timespan it gained lots of new capabilities. Extrapolate, don't just look at what they can do today.
Atoms are more difficult than bits, so physical applications (e.g. in factories, robotics) will lag. I also expect specialist models like AlphaFold slowly finding their niches over time.
> significant, and one that opens up lots of new interesting applications as well as invites lots more R&D — but like the neural net it’s not gonna fundamentally transform society or industry.
Yeah, and semiconductor electronics that grad-students built from of germanium by painstakingly pressing them together are very interesting and worth additional R&D but won't transform society either.
I guess where both you and the GP are correct is that you can't make an autonomous car by applying transformers exclusively.
LLMs are "language I/O interfaces", and as such there isn't a lot of value they can create alone (even most of the currently proposed uses are bullshit). But it's quite likely that they'll be there on a lot of advanced technology helping it do what you meant.
Still even though both will be there in the transformative technology, and even though the technology may not even be useful without them, they are not central pieces on most of it.
llm companies raise lots of money on crazy evaluation, the question is if they will be able to build enough revenue stream to justify further money injections.
No you just decide what movie you want to watch and pay $4 to rent it on demand. Very common experience available to anyone with a smart TV or connected streaming device.
VR covers your entire field of view, so not sure why you’d claim that it precludes multitasking, even if the current iterations aren’t yet geared toward that.
I think the companies pursuing VR see that as a momentary technical deficiency, not an inherent limitation of VR generally. So to characterize their efforts as a long-term “bet” against multitasking seems silly to me.
It appears that this was just a misreading of how memory usage was being reported and there was actually no improvement here. At least nothing so sensational as being able to run a larger-than-RAM model without swapping from disk on every iteration.
Please read the original link to the pull request, where I stated my change offered a 2x improvement in memory usage. You actually are able to load models 2x larger without compromising system stability, because pages are no longer being copied. That's because you previously needed 40gb of RAM to load a 20GB model, in order to ensure your file cache wasn't destroyed and need to reread from disk the next time. Now you only need 20GB to load a 20GB model.
The peculiarity here is that tools like htop were reporting the improvement as being an 8x improvement, which is interesting, because RAM use is only 2x better due to my change. The rusage.com page fault reporting was also interesting too. This is not due to sparseness. It's because htop was subtracting MAP_SHARED memory. The htop docs say on my computer that the color purple is used to display shared memory, and yellow is used to display kernel file caches. But it turned out it just uses yellow for both, even though it shouldn't, because mincore() reported that the shared memory had been loaded into the resident set size.
It's obviously a productive change and kudos for taking it on, but much of the enthusiasm being generated here was driven by the entirely unanticipated prospect of running a model at full speed using less memory than the model's own footprint, and by the notion that inference with a dense model somehow behaved in a sparse manner at runtime. Best to be a bit more grounded here, particularly with regard to claims that defy common understanding.
I wanted it to be sparse. Doesn't matter if it wasn't. We're already talking about how to modify the training and evaluation to make it sparser. That's the next logical breakthrough in getting inference for larger models running on tinier machines. If you think I haven't done enough to encourage skepticism, then I'd remind you that we all share the same dream of being able to run these large language models on our own. I can't control how people feel. Especially not when the numbers reported by our tools are telling us what we want to be true.
The debate over what kind of intelligence these models possess is rightly lively and ongoing.
It’s clear that at the least, they can decipher very numerous patterns across a wide range of conceptual depths — it’s an architectural advance easily on the the level of the convolutional neural network, if not even more profound. The idea that NLP is “solved” isn’t a crazy notion, though I won’t take a side on that.
That said, it’s equally obvious that they are not AGI unless you have a really uninspired and self-limiting definition of AGI. They are purely feedforward aside from the single generated token that becomes part of the input to the next iteration. Multimodality has not been incorporated (aside from possibly a limited form in GPT-4). Real-world decision-making and agency is entirely outside the bounds of what these models can conceive or act towards.
Effectively and by design these models are computational behemoths trained to do one singular task only — wring a large textual input though an enormous interconnected web of calculations purely in service of distilling everything down to a single word as output, a hopefully plausible guess at what’s next given what’s been seen.
AGI is Artificial General Intelligence. We have absolutely passed the bar of artificial and generally intelligent. It's not my fault goal post shifting is rampant in this field.
And you want to know the crazier thing? Evidently a lot of researchers feel similarly too.
General Purpose Technologies ( from the Jobs Paper), General Artificial Intelligence (from the creativity paper). Want to know the original title of the recent Microsoft paper ? "First contact with an AGI system".
The skirting around the word that is now happening is insanely funny. Look at the last one. Fuck, they just switched the word order. Nobody wants to call a spade a spade yet but it's obvious people are figuring it out.
I can you show you output that clearly demonstrates understanding and reasoning. That's not the problem. The problem is that when I do, the argument Quickly shifts to "it's not true understanding!"
What a bizzare argument.
This is the fallacy of the philosophical zombie. Somehow there is this extra special distinction between two things and yet you can't actually show it. You can't test for so called huge distinction. A distinction that can't be tested for is not a distinction.
The intelligence arguments are also stupid because they miss the point entirely.
What matters is that the plane still flies, the car still drives and the boat still sails.
For the people who are now salivating at their potential, or dreading the possibility of being made redundant by them, these large language models are already intelligent enough to matter.
> ... these large language models are already intelligent enough to matter.
I'm definitely not contesting that.
I've always considered the idea of "AGI" to mean something of the holy grail of machine learning -- the point at which there is no real point in pursuing further advances in artificial intelligence because the AI itself will discover and apply such augmentations using its own capabilities.
I have seen no evidence that these transformer models would be able to do this, but if the current models can do so do then perhaps I will eat my words. (Doing this would likely mean that GPT-4 would need to propose, implement, and empirically test some fundamental architectural advancements in both multimodal and reinforcement learning.)
By the way, many researchers are equally convinced that these models are in fact not AGI -- that includes the head of OpenAI.
See what you're describing is much closer to ASI. At least, it used to be. This is the big problem I have. The constant post shifting is maddening.
AGI went from meaning Generally Intelligent to as smart as Human experts and then now smarter than all experts combined. You'll forgive me if I no longer want to play this game.
I know some researchers disagree. That's fine. The point I was really getting at is that no researcher worth his salt can call these models narrow anymore. There's absolutely nothing narrow about GPT and the like. So if you think it's not AGI, you've come to accept it no longer means general intelligence.
>> The point I was really getting at is that no researcher worth his salt can call these models narrow anymore.
Are you talking about large language models (LLMs)? Because those are narrow, and brittle, and dumb as bricks, and I don't care a jot about your "No True Scotsman". LLMs can only operate on text, they can only output text that demonstrates "reasoning" when their training text has instances of text detailing the solutions of reasoning problems similar to the ones they're asked to solve, and their output depends entirely on their input: you change the prompt and the "AGI" becomes a drooling idiot, and v.v.
That's no sign of intelligence and you should re-evaluate your unbridled enthusiasm. You believe in magick, and you are loudly proclaiming your belief in magick. Examples abound in history that magick doesn't work, and only science does.
I've been using chatgpt for a day and determined it absolutely can reason.
I'm an old hat hobby programmer that played around with ai demos back in the mid to late 90s and 2000s and chatgpt is nothing like any ai I've ever seen before.
It absolutely can appear to reason especially if you manipulate it out of its safety controls.
I don't know what it's doing to cause such compelling output, but it's certainly not just recursively spitting out good words to use next.
That said, there are fundamental problems with chatgpt's understanding of reality, which is to say it's about as knowledgeable as a box of rocks. Or perhaps a better analogy is about as smart as a room sized pile of loose papers.
But knowing about reality and reasoning are two very different things.
Have you tried out gpt4? If not and you can get access I'd really recommend it. It's drastically better than what you get on the free version - probably only a little on the absolute scale of intelligence but then so is the difference between an average person and a smart person is small on the scale from "worm" to "supergenius".
This is convenient behavior up until you actually have an incident that coincides with theirs, in which case it becomes catastrophic because you had no idea that outside vigilance was required on account of their ingestion downtime. Not sure why you would laud this. Is it possible to opt out?
In your scenario you would have no logs etc until the DD incident resolved.
Opting out would just mean all your missing data alerts fire every time Datadog has an incident and you would then check, see that everything is missing, and then identify the cause as the Datadog incident.
Its much better to have them handle it and auto-mute the impacted monitors than communicate to my customers every time about false alerts saying all our services are down.
> Opting out would just mean all your missing data alerts fire every time Datadog has an incident and you would then check, see that everything is missing, and then identify the cause as the Datadog incident.
You are missing the last step, which is that, knowing alerts are down, you can actively monitor using other tools/reporting for the duration of their incident.
And why would you have no logs? Even assuming you ingest logs through Datadog (they monitor on much than just logs and not everyone uses all facets of their offering), you would presumably have some way to access them more directly (even tailing output directly if necessary).
And lastly, why would you communicate to your customers without any idea of the scope or cause of the issue? It would likely be clear very quickly that Datadog was having issues when you see that all your metrics are suddenly discontinued without other ill effect.
>knowing alerts are down, you can actively monitor using other tools/reporting for the duration of their incident.
If you just want notifications for when datadog is down, their StatusPage does a fine job of clearly communicating incidents.
I wouldn't want to rely on a "when multiple of our 'missing business metric' monitors alert, check and see if datadog is down" step in a runbook. I don't like false alerts. I don't like paging folks about false alerts. Waking up an oncall dev at 2am saying all of production is down when it is just datadog is bad for morale. Alert fatigue is a real and measurable issue with consequences. Avoiding false alerts is good. If the notification says "all of production is down" and that isn't the case, there is impact for that. I'd much prefer having a StatusPage alert at a lower severity and communication level say "datadog ingestion is down".
Instead, use their StatusPage notifications and then execute your plan from that notification, not all of your alerts firing.
>And why would you have no logs?
I mean Datadog logs/metrics etc. Currently, we are missing everything from them. We can still ssh into things etc, they aren't gone, but from the Datadog monitor's view in this scenario, they stopped seeing logs/metrics and would alert if Datadog didn't automatically mute them.
>why would you communicate to your customers without any idea of the scope or cause of the issue?
We prioritize Time To Communicate as a metric. When we notice and issue in production, we want customers to find out from us that we are investigating instead of troubleshooting and encountering the issue themselves, getting mad, and clogging up our support resources. Flaky alerts here don't work at all for us.
But it illustrates the general dissatisfaction among TikTok users with the other mainstream US social content platforms.