> "Chollet thinks he knows what’s going on: summer vacation... a significant portion of students using ChatGPT to do their homework. It’s one of the most common uses for ChatGPT, according to Sam Gilbert, a data scientist and author."
Given that ChatGPT was just released last fall and it was expected to revolutionize practically everything, isn't it still telling that usage is falling? Where are the scientists, lawyers, doctors, office workers etc? You'd expect a slowing in the rate of growth, not shrinking.
I don't know, if high school and undergrad students being off for the summer is enough to shrink your app maybe it's not all that?
Edit: I understand there are new tools coming, but most of them aren't out yet. GPT4 was just released. For the most part, if you want to play with AI, ChatGPT is it. If they're really experiencing a decline in users that's not great for them.
I see people using ChatGPT, Bard, etc. to answer questions on mailing list threads. It'll be a long thread of thoughtful human written responses, and then someone will come along and say "I asked Bard what it thought about this issue, here's the response". And then out comes five paragraphs of text that I might as well not even bother reading, as it often contains falsehoods that I have to waste time looking up and double-checking. It bothers me so much that these LLMs don't have any inherent understanding of truth or facts, so the only thing they're really good at is writing fiction (e.g. I hear they're pretty good as a DM's assistant to flesh out flavor/scenario descriptions and such). The Midjourney generated images fall into the same fiction category and are generally pretty good too.
But for actual non-fiction usage, you have to spend so much time triple-checking everything they say to ensure that it isn't simply complete nonsense. What's the point?
The point is you need to collaborate with the LLM, not just expect it to provide fully-formed answers. I've saved countless hours of work over the past few months using GPT-4 to help me solve problems. One general framework can be to use it to help you google a solution instead of a problem. Googling a problem gives you all kinds of results, which you then need to dig through and understand in order to evaluate whether there's something that can be adapted to your use case. On the other hand, you can ask GPT-4 for solutions to the problem, and then google those, which cuts the process down significantly. Of course that's only one example. The pattern though is that you need to play to its strengths. It's not particularly smart, but it's very knowledgeable. Just like a human though, you can't expect it to perfectly regurgitate all the details. You use it for direction, and then refine from there, through conversation with the LLM as well as external research.
It really is astonishing how much you can get done this way. I've been setting up a home lab for myself, and the answers Gpt4 gives are miles ahead of the stack overflow results or documentation of the apps or whatever else. Rarely (very rarely) it will give me a wrong answer, but then I paste in the error message or describe the problem I had and it almost always comes up with the correct answer 2nd try. Final step is asking where I might learn more because it's not working, and gpt always gives me a better link than google.
I'm convinced the people who say it's nothing but a BS machine have never tried to use it step by step for a project. Or they tried to use it for a project most humans couldn't do, and got upset when it was only 95% perfect.
I disagree with that. It's very useful writing boilerplate and documentation, but two third the time I'm in front of a bug I'm to lazy to understand and ask ChatGPT, with context and all, the answer is wrong. I can fiddle with it to reduce that to a third of the time, but in the end, only the questions that are really, really hard to figure on your own are left.
Still, it's way better and more efficient than Google. Less than not being lazy and using my two braincells tbh.
My newest use is
Hello, i' m working on X, I use Y tech, my app do Z and I want to implement W. Can you provide a plan on how and where to start?
I agree with this. This is my primary use as a new analyst. Weird things that would take lots of time to dig through stack overflow to find, I can find pretty quickly if I feed it the parameters I’m working within, and what I’m trying to get to. Usually it just fills in the gap that Google was doing before, but much better in my opinion.
Sorry i'm a bit late. Depends. Professionally, its a mix a python, typescript (those i practically never use ChatGPT for, or rather, i use it for questions i usually ask google/reddit/SO), terraform/terragrunt on AWS with some Cisco config and some other hardware stack i don't remember but that require custom terraform providers. I automate the deployment of the hardware, so i think writing custom providers and terraform is roughly a third of what i do and i cannot use ChatGPT for that, its output is way too bad.
Personnally, a lot of bash, C, AWK at the moment (typescript + html/css until last april, now i'm back to the basics). The figure i gave in my post were more for that.
The last time i used it was yesterday, i wanted to hack something on an old game i used steam+proton for. I knew it was a weird Wineprefix, so i asked ChatGPT for it, i might have asked poorly, but after fiddling, i had the response (tbh i had to look how to get the game ID, so in the end i lost more time than not), then when it still didn't work, cause the path was shit, i entered all necessary context in ChaGPT4, and it couldn't find the easy "USER=steamuser" env variable to add before launching Wine. I stopped after 10 minutes, looked into a example Wine cfg file, understood the issue and fixed the problem myself.
I mean, it's probably good for really basic stuff, so it could have helped me when i was starting, but 80% of the stuff i code automatically without really thinking about it, and when i have to stop to think, ChatGPT isn't helping. Also tbh, VSCode is really, really good and fix my old, time-consuming task of "what's this argument again?"
Oh come on. I fed Unreal c++ engine code to ChatGPT4 and it couldn't understand inheritence in Slate classes and therefore kept offering me the same broken solution for a parameter with the wrong type.
The Unreal engine code is documented and publicly avaiable for OpenAI to ingest and it still gets the basics wrong.
I wasted hours trying to get it to explain to me what I didn't know, if it doesn't understand the internals of Unreal, I have no hope for it on bigger and better codebases.
It doesn't parse, it doesn't explain, it does not grok. It guesses at best and the blood sucking robot-horse is not telling the truth.
>It doesn't parse, it doesn't explain, it does not grok. It guesses at best and the blood sucking robot-horse is not telling the truth.
In my experience with coding (I've only done javascript and python myself) you have to tell it to explain and grok. It takes on the role you give it. Even just saying something like "you are a professional unreal developer specializing in C++, I am your apprentice writing code to (x). I want you to parse the following code in chunks, and tell me what might be wrong with it" before typing your prompt can help the output immensely. It starts to parse things because it's taken on the role of a teacher.
People love to hate on the idea of "prompt engineering" but it really is important how you prime the thing before asking it a question. The other thing I do is feed it the code slowly, and in logical steps. Feeding it 20 lines of code with a particular purpose / question you'll get a much better answer than feeding 200 lines of code with "what's wrong here?" You still need to know 90% of what's going on, and it becomes very good at helping out with that 10% you're missing. But for all I know it is just really bad at C++, that wouldn't surprise me. The things I'm using it for are definitely more simple.
I do think this is why I sometimes get amazing results, and other times I have to go over a snippet of code so often I just give up and do it myself. It's a matter of how the question was asked in the first place.
Knowing that, it makes sense that your prompt should be as specific as possible if you want the results to be as specific as possible.
The best results I got was feeding it Lisp code that I wanted translated to C (to compile it). It took very little effort on my part because I described what each of the snippets did separately, and the expectation when combined and used together.
Through this, I learned that C doesn't have anything akin to the Lisp's (ATOM). ChatGPT stated clearly that its version of ATOM should only be expected to work in the code it was writing, but might not work as expected if copied out for another use of Lisp's (ATOM).
I asked it to give examples of where it wouldn't work, and it gave me an example of a code snippet that used (ATOM) that would not have worked correctly with the snippet that did work correctly with my original purpose.
Having said that, I myself learned that working with code function by function with ChatGPT, and being explicit about what you need, gives very good results. Focusing on too many things at one time can derail the whole session. One or two intermingling functions works great though.
GPT4 works best when you assume that you're the professional dev with decades of experience, whereas GPT4 is a bright and broadly-informed co-op student lacking in experience in getting stuff working. You have to have a solution in mind, and coach it with specifics. And recognize the tipping point where it takes you more keystrokes of English to say what should be done, than keystrokes in Vim to do it yourself.
I did prompt engineer, using the 'you are an expert, desribe to student with examples' in many different variations.
In my testing prompts did not unlock an ability in GPT to grok the structure of code.
Empirical testing of LLM's is going to prove and map out it's weaknesses.
It is wise to infer from intution and examples what it can handle, leave the empirical map of it's capabilities to the academics, for the provable conclusions.
My observation (which could be wrong) is that ChatGPT as a programmer's aid is only useful for the simple cases. Not so much for complex stuff, and certainly not for something as complex as the Unreal engine.
Do you have some sample chat logs of interactions like this you can share? I'm curious to see what kind of stuff it's coming up with, and how you're prompting it.
I don't tend to keep the chat logs, as the amount of them gets unwieldy very quickly. But examples of things I've done with it that are useful:
I wanted to create a web app, something I haven't done in a very long time. Just a simple throwaway back-of-the napkin app for personal use. I described what I wanted it to do, and asked what might be a good frontend/backend. It listed a few, I narrowed it down even more. Ended up deciding on flask/quasar.
After helping me setup VS Code with the proper extensions for fancy editing, and guiding me through the basic quasar/flask setup, it then was able to help me immensely creating a basic login page for the app. Then it easily integrated openAI api into it with all the proper quasar sliders for tokens/temperature/etc. Then it created a pretty good CSS template for the app as well, and a color scheme that I was able to describe as "something on adobe color that is professional and x and x (friendly, warm, whatever you want to put in)". Everything worked flawlessly with very little fuss, and I'd never used flask or quasar before in my life. You can also delve VERY deep into how to make the app more secure, as I did for fun one evening even though it's not going to be internet facing.
Another thing I did was go over some pfSense documentation with it. I had some clarifying questions about HAProxy, as well as setting up Acme Certificates with my specific DNS provider. It was extremely helpful with both. It also taught me about nitty gritty settings in the Unbound DNS resolver in a way that's much more informative than the documentation, and helped me set up some internal domains for pihole, xen orchestra, etc with certificates. Also helped me separate out my networks (IoT, Guest network, etc), and taught me about Avahi to access my hue lights through mDNS.These are things I always wanted to do, I just never felt like going down a google rabbit hole getting mostly the wrong answers.
Last example I'll give is it was able to help me set up docker-compose plex within portainer that then uses my nvidia GPU for acceleration. The only thing I had to change from the instructions it gave was to get updated nvidia driver #s and I grabbed the latest docker-compose file. I'd never used portainer in my life before, nor do I have experience with nvidia drivers within linux, and I feel like learning it was many times faster being able to ask a chatbot question vs trying to google everything. Granted I still had to RTFM for the basics, as everyone should always do.
I think perhaps my use cases are a bit more "basic" than many HN users. Like I said I'm not asking it to do problems most humans wouldn't be able to do, as I know it isn't quite there yet. But for things like XCP-ng, portainer, linux scripts, learning software you've never used before, or even just framing a problem I'm having in steps I hadn't thought of it's been invaluable to me. For me it's like documentation you can ask clarifying questions to. And almost none of the things I've asked it would work at all if it were wrong, I would know immediately.
>Googling a problem gives you all kinds of results, which you then need to dig through and understand in order to evaluate whether there's something that can be adapted to your use case.
Exactly; search engines give you those blue links and short descriptions of the search results which are not enough for you to grasp what is the website about. I think what search engines need to do is tackle the complexity of going through the results of a search engine. Google page rank seemed like a silver bullet back in the day but the websites which are the most popular are not necessarily of the best quality. What we need is to lower the complexity for casual users when they deal with search results.
On the other hand ChatGPT is like an answer machine that can give you satisfactory answer on your fist try but if not, you need to talk with it, push it and explore what answers it gives you, just like you said. I think ChatGPT type search engine will be more suitable for people who are "lazy" or for the people who don't have time to "Google" and go through search results and look around the web for the helpful and useful information.
> I think what search engines need to do is tackle the complexity of going through the results of a search engine.
This is exactly what I don't want a search engine to do for me. Going through the list of results and evaluating them is an important part of my process, if what I'm trying to do is learn something new.
So how do you differentiate blue links that all look the same? I mean yes there is description of a search result but that doesn't tell you very much about the website or the quality of information that you are getting. Bing did a good thing with their annotations[0] because my thinking is that something like annotations lower the complexity of browsing and skimming through search results.
> So how do you differentiate blue links that all look the same?
They don't all look the same. They all tend to go to different places. I find that it's reasonably easy to spot a great deal of garbage sites just from their domain name or url, and that weeds out a large chunk. Ignoring multiple results for the same site also weeds out a large chunk (I only need one of them).
The rest, I just click on and take a look at the page. It's pretty quick and easy to weed out most of the garbage ones with a quick skim.
The rest, I sample, read captions and boxes, skim paragraphs and such to determine if it's along the lines of what I want. That's pretty quick too.
For the most part, it's the same process that you use when researching in a library.
The reason that I want to do this myself rather than outsourcing it is because I'll inevitably learn something in the process that will shift my viewpoint to one that's more targeted or meaningful for the purpose I have in searching.
It doesn't matter how good the engine is at collating and summarizing results -- even if it's perfect, my understanding not only of what I'm looking to learn, but also discovery of important but serendipitous or unexpected knowledge, is lessened.
It's a bit like the difference between reading Cliff's (or Cole's) Notes about a book and reading the book.
I've tried ChatGPT inside Microsoft's Bing search engine and it is pretty good so far. It definitely saved me time that I would lose clicking on search results and skimming through websites looking for useful information.
So many small things go faster. For example, I throw the output of Windows-Shift-T (power tools keyboard shortcut for screenshot OCR) text into ChatGPT with a "remove line breaks" prompt. Yes, I know how to Ctrl-H ^l in Word, and other ways, but they sometimes produce odd results (missing spaces, extra spaces), and GPT is faster.
I'm getting people filing tickets for... let's call them complex, medium-large projects that they want implemented. Helpfully, they're including ChatGPT "instructions" for how to do it. I got my first "but ChatGPT said..." argument about why I'm wrong about something just yesterday.
Somewhat more general, but I've pretty much already decided that if I find people using it to talk to me without telling me, I won't be talking to them. Goes for businesses as well as personal things - don't gaslight me, or you will lose the option to do so.
As a fiction author, I don’t feel gpt writes good fiction at all. It’s a non-fiction tool, just one that lies to you constantly. Half of what it says is a lie…
Yes, it’s terrible at writing fiction. Cliché laden, dull, repetitive prose and somehow, in terms of actual content, it never generates anything of interest. Its desire to always wrap things up into a happy ending within a couple of paragraphs also means it’s almost incapable of generating conflict. At best it can generate teenage fan fiction — and bad teenage fan fiction, at that.
Why do people use bard, like genuinely, it just gives way worse responses than bing esp on creative mode, the only good thing bard has going for it is that the frontend programmers knew what they were doing and it doesn't auto scroll up when it types.
> And then out comes five paragraphs of text that I might as well not even bother reading, as it often contains falsehoods that I have to waste time looking up and double-checking.
I'm curious on if you feel human generated content does not contain falsehoods.
I consider code to be non-fiction, and ChatGPT will generate stuff that compiles and works most of the time. There's no need to triple-check the code output.
"Code" is really a much much much smaller and much much much more structured output than "English words".
Presumably, the system was trained with a very small amount of "untrue code" in the sense of stuff that just absolutely could never work. And also presumably, it was trained with a lot of free-form text that was definitely wrong or false, and highly likely to have been originally created to be purposefully misleading, or at a minimum, fiction.
That the system outputs reliable code tells us nothing about its current ability to output highly reliable free form text.
But it understands English words and translates the meaning behind them into code very well, particularly with a bit of iteration. Simply taking the interface from writing code to speaking in plain language is a huge practical accomplishment.
> Given that ChatGPT was just released last fall and it was expected to revolutionize practically everything, isn't it still telling that usage is falling?
Its not, though. ChatGPT, the website is basically an (increasingly non-exclusive, given BingAI) frontend. What is claimed to be revolutionary is the underlying models (in the narrow view) and similar generative AI systems (in the broader view). As more applications are built with either OpenAIs own underlying models or those from the broader space, the ChatGPT website should be expected to represent a smaller share of the relevant universe of use.
This is the correct viewpoint. ChatGPT is one specific implementation of the technology, which was most people's first exposure to it. The broader applications of the technology itself are still in the very early stages, but are already making significant impacts.
Honestly does it really matter much if their B2C usage is primarily students?
I’m more curious about their B2B operations which is likely what will trickle into more people’s lives. Their APIs seem to enable quite a few interesting possibilities with a low up front technical investment.
Too me the whole “generate me a bunch of text and display it as text” is a niche use case for a lot of people. Integrations for web search, document search, summarization, data extraction/transformation, and sentiment analysis are more useful and less likely to have hallucinations affect the end product.
Regarding their revenue I’m curious how the Azure hosted OpenAI services work for OpenAI. Billing is all through Microsoft and the documentation tries really hard to make it clear these are Azure services. I wonder if Microsoft just pays a licensing fee or if there is some revenue sharing going on.
> Honestly does it really matter much if their B2C usage is primarily students?
For ChatGPT specifically, yes, because it keeps getting hyped so much, and I think B2C is going to be what ChatGPT ends up with. My company isn't known for its tech innovation and we're spinning up our own LLM based on our own data. It doesn't need to be super fast because I doubt we're going to go the chatbot route. It will likely be for content generation, so less horsepower is fine. No need to pay OpenAI for excess capacity.
They could go public now with an unreal valuation based on the hype of B2B usage that may never materialize. Revealing that you're mainly a cheat/study tool for students puts you in a box with Chegg and others, at a much lower valuation.
> If they're really experiencing a decline in users that's not great for them.
If it's the case that "there are new tools coming, but most of them aren't out yet" - and I believe it is[0] - then the overall userbase of ChatGPT doesn't matter to OpenAI, because soon enough the same models will come back with a vengeance, in a different, more streamlined form.
In fact, I feel that the major change will happen if and when Microsoft gets their Office 365 and Windows copilots working and properly released: they'll have instant penetration into every industry, including scientists, lawyers, doctors, and office workers.
--
[0] - It's been only few months. Between playing around, experimenting, then developing, testing and marketing a tool, there just hasn't been enough time to do all of that.
I've always seen it as a toy, programed by humans, with human fallacies. It's a fun search engine for sure, but I think it's a decade (at least) until it becomes practical and useful.
I see it as multiple things combining to reach a critical mass. Yes it was the killer app but it was also the first time it was actually good enough to chat with. Put another way- it's the shittiest LLM application going forward or the worst LLM application that people are willing to pay for. The hype isn't specifically for ChatGPT but imagining the trendline of LLMs or LLM adjacent tech projected forward combined with still falling compute cost and further democratization.
There is likely also a ton of users who were just playing with it and had no true use case beyond asking it to make haikus about cats and got bored. IMO their true product is and always has been the API.
It was about time to build a new desktop anyways (roughly 4 to 6 years before the old one goes to frolic at the server farm in the basement) and $2,000 will easily buy a machine that can run the quantized 65b models right now. So I spent slightly more than I normally do on this latest box and it's happily spitting out 10+ tokens a second.
You're not going to beat GPT-4 yet, but you have direct control over where your info goes, what model you're running, compliance with work policies against using public AI, and relatively cheap fixed costs.
Not to mention, the local version works with no internet and isn't subject to provider outages (not entirely true - but you're the provider and can resolve).
Seems like an easy win for anyone who might be buying a desktop for graphic/gaming anyways.
My experience with open source models places them as a little bit worse than GPT 3, and nowhere close to GPT3.5.
That said:
- For many uses, it doesn't matter. For many of the ways I use it, I don't care. For basic use (e.g. clean up an email for me), it's basically the same. For things like complex reasoning, algorithms, or foreign languages, the hosted service is critical.
- GPT3-grade models have more soul. OpenAI trained GPT3.5 and 4 to never do anything offensive, and that has a lot of negative side effects, well-documented in research. The way I'd describe it, though, is the difference between talking to a call center rep and your grandma (with mild Alzheimer's, perhaps). They both have their place.
- Different models are often helpful in workflows.
My experience is anecdotal. Please don't take it as more than one data point. If other people post their anecdotal experiences, you'll get the plural of "anecdote."
Would you write a check to fund a business that could potentially self-destruct via lawsuits alone? In the end, the best model will not be owned by a mega corp like MicrOpenAI. It may be the most popular, but it will be the equivalent of the sanitized version of history students learn in school. The best model will have no problem telling you, very factually, that the hallways of Versailles used to smell like sh--.
If you're a diligent tester, none of the open source models can touch GPT 3.5 yet, however, in practical terms, some of the 60b and 30b parameter models are almost indistinguishable from GPT 3.5 from the layperson's perspective. Now, if you consider the uncensored models, then you actually have some capabilities that GPT 3.5 and 4 are completely lacking.
Based on the rate of progress in the open source world, it won't be more than a year before we have an open source model that is truly superior to GPT 3.5
The commercial & api based models are still more capable general purpose tools. But the current open tooling can do some nifty stuff, and the community around it is moving at a breakneck speed still.
In some areas, it's acceptably good. In some areas it's not. But it's getting better really fast.
That's why my current plan is to get ChatGPT 4 to help me set up my local open source implementations of Orca and Stable Diffusion. Got MusicGen running locally anyway; that was pretty easy.
It depends heavily on the model you're running, and to some extent what you're doing with it. It also depends to on prompt effort. The quantized llama 65b model (you can do it yourself, or pull something like https://huggingface.co/TheBloke/llama-65B-GGML) is probably the highest quality for general purpose, but it does take a fair bit of effort to prompt since it's not tuned for a use-case.
It's also not licensed commercially, so I avoid some things with it (ex: I do a lot of personal learning/investigation, but it doesn't touch or write anything related to work or personal projects)
The open models are a little further behind, but it's interesting to see them spin off into niches where they have strengths based on tuning/training.
I don’t know about self-hosted, but it the company I contact at has 3.5 instance. It has better understanding of code examples I provide than ChatGPT. The company hasn’t tweaked the code to our standards.
2. Make sure you have the latest nvidia driver for your machine, along with the cuda toolkit. This will vary by OS but is fairly easy on most linux distros.
4. Run the model following their instructions. There are several flags that are important, but you can also just use their server example that was added a few days ago - it gives a fairly solid chat interface.
4) Run gpt4all, and wait for the obnoxiously slow startup time
... and that's it. On my machine, it works perfectly well -- about as fast as the web service version of GPT. I have a decent GPU, but I never checked if it's using it, since it's fast enough.
Some example models (I'm linking to quantized versions that someone else has made, but the tooling is in the above repos to create them from the published fp16 models)
In hindsight - I don't know that the second GPU was worth the spend. The c++ tooling is doing a very good job right now at spreading work between GPU vram and main ram and still being fast enough. Even ~4/5 tokens a second is fast enough to not feel like you're waiting.
I'd suggest skipping the second card and dropping the price quite a bit (~2100 vs ~2900) unless you want to tune/train models.
Google Cloud for both VertexAI & "self-hosted" (run any model on GPUs). We are already there with the rest of our cloud, better privacy when you use gcloud apis than OpenAI
While Francois Chollet is not actually an employee of OpenAI, and while that article simply quotes a tweet he made rather than a substantive interview .. .. this is still clearly a big problem for OpenAI as students trying to cheat on exams is a highly profitable $1T industry that Microsoft badly needs to disrupt.
If it meant good grades without doing the work and it became socially acceptable it is ~150 million global tertiary students times 10 months of the year times whatever that is worth on a monthly basis. Maybe only $150B ARR right now at $100/mo (textbooks to read cost more than this, but you don’t need them anymore), so not $1T on students but still massive, and if you capture them in pre/early career it is easily $1T in ARR once your queue deepens.
So how do we value ChatGPT then? Should it be as valuable as Coursera, Udacity etc put together? Should it be that PLUS Harvard Deloitte and ServiceNow? Or since they have no major moat should it be Zero? It’s hype of this scale that can rationalize unprecedented figures.
I should make explicit that I think it is impossible to achieve in practice because higher education will collapse completely before they achieve any fraction of market saturation with the numbers I wrote.
Coursera is a discount retailer of tertiary education so I don’t see any relevant connection.
ChatGPT/OpenAI specifically is probably already overvalued unless they have some first-mover advantage that I can’t see.
Someone else has said it many times over but I agree that LLMs will disappear into the technology stack — a rising tide that lifts many market caps.
But that’s part of the point: ChatGPT is so hyped it bends perception. The OP article after all quoted a Google employee who has nothing to do with ChatGPT.
> Found out later she used ChatGPT to give her a summary of the first chapters. She did not enjoy having first weeks of summer without a phone.
Honestly, the pupils of my generation had other sources for finding summaries of the books to read for school (e.g. on the web, or in form of other books that contain a summary of the book to read (the respective pages were copied and these copies were passed among classmates).
Thus: just the methods change over generations, how pupils behave is much more constant.
Yes, but other than being conditioned to believe the experience of reading is superior, why are we striving for that experience?
Doing the square root of 147 by hand is a more educational experience than using a calculator, but I can understand the concept of a square root without having to factorize 147 (the same cannot be said of my pre-calculator ancestors).
It also saves me from having to remember the decimal representation of sqrt3.
Reading helps build vocabulary and probably helps improve her use of the language as a whole. It’s like running to lose weight but you decide to cheat by taking a cab instead.
With math, when you are first learning the very basics it helps to do a few problems by hand to get a feel of things. Sure if you have a degree in STEM have have deal with advance math you can probably get away with just reading the theory only.
My argument is that reading is actually a very bad way to learn/communicate and always has been.
We just built a very strong culture encouraging it because for a long time it was the most efficient way to do both things.
Now it’s not, but the tradition of it being important is hard to shake.
The closest analogue in my mind is how the classics in their original Greek were considered the only true way to learn for a very long time (even once most of the important knowledge had been translated to English very well).
Reading is a perfectly fine way to increase one’s vocabulary. It’s not the only way - TV and actually conversation would work too. I don’t see why it’s a bad way. It’s cheap and doesn’t require manpower per session.
For some people, it still is the most efficient way to learn/communicate. Nothing is more annoying then using google to try to solve some problem and the first few results are videos...
The world is changing, and children should change with it. Learn and understand technology, and the benefits of where modern machine learning is taking us.
But you cannot unironically think we can substitute fundamental skills like essay writing and critical thinking for a degree in 'prompt engineering'?
> But you cannot unironically think we can substitute fundamental skills like essay writing and critical thinking for a degree in 'prompt engineering'?
Personally, I don't think that. However, I also think it's logical and good for a child to consider the task at hand and select the most efficient tool available.
I am not this girl's parent, and I'm not sure how I would have handled the situation if I was. However, I worry that simply taking away her phone may have been counterproductive. I would have erred toward some sort of open conversation about the purpose of the assignment and what she is hoping to learn.
I don't know what the best practice is. I'm not a parent yet but I'm studying to be an elementary school teacher, so these sorts of questions are very much on my mind.
Essay writing has already become an old world archaic skill in my mind, like a seamstress or blacksmith. People still produce clothing and metal tools, but with 1000x fewer labor hours for the same amount of output and the process looks completely different. Even most of my personal messages get LLMed if they need to be very long or have to explain something complicated.
Critical thinking is still a skill, but even there GPT-4 is great at giving suggestions and ideas when you're not sure what to do. I got into a long recurring argument with extended family members at an event last month. I pulled out my laptop and dropped in everyone's complaints into GPT and asked the best way to move forward. An argument that normally would have gone on for hours was over in 20 minutes, and we actually felt a sense of mutual understanding for once.
"How about accepting new realities and changing homework assignments instead?"
Suggestions for how to change homework assignments? It seems like a difficult problem, so reverting to hand writing things with no phones is probably the best option for now...in my opinion.
1. As you note, consider eliminating it, or replacing it with in-class or lab work if it's absolutely necessary.
2. Consider making it non-graded (optionally providing feedback if desired that will not be counted in course grading.)
3. Require oral/in-person explanation of homework to paid peer assistants recruited from the previous cohort.
4. For writing, change to a writing workshop model where ChatGPT is a permitted tool that is incorporated into the workshop and students can learn how it can be helpful and how it can run astray. Help students to find and write in their own distinctive voice, even if they are assisted by ChatGPT.
Have students work with ChatGPT and let them spot / mark where the AI is hallucinating or wrong? Might translate well to general competence wrt working with any kind of source online.
It is more so the fact that someone takes something from ChatGPT, and then just feeds potentially contentious assertions into Google to see if they are true or not.
What does someone learn from doing this? What happens when they actually have to use their brain to do something hard?
If someone plugs math into a calculator, are they really learning something when it spits out an answer?
Of course! How to apply the calculator to problems, and how to discern if a problem is calculator-solveable. We've decided as a society it's better for students to use calculators as a tool in mathematics, why is chatGPT different in literature?
I don't know how calculators are used in classrooms today, but when I was going through school, the general idea was that we could use calculators to do tasks that we had already learned to do.
E.g., we could not use a calculator at all when learning arithmetic.
Once we learned arithmetic and were learning (say) algebra, we could use the calculator to do arithmetic, but not to compute the algebra for us.
While learning trigonometry, we could use the calculator for arithmetic, but not for computing sines and cosines.
While learning calculus, we could use the calculator for arithmetic and sines and cosines, but we could not use it to perform integration or differentiation for us.
You can't type your homework into a calculator and get the answer out (at least not after basic arithmetic).
You have to understand the problem and how to use the tool.
If one's understanding of how to use the tool is limited to "access the tool and dump in the question, then blindly paste the response into my homework document", I'd say you are not learning much. It's obvious that every contemporary teenager can learn that in a few minutes. That's not giving us the capabilities to move forward as a society.
As anecdote, I just cancelled my subscription to ChatGPT a few days ago.
It was fun to experiment with, but it’s obvious that 90% of their development effort has been going into censoring their models instead of improving their utility. I saw no tangible improvements over months.
For example, the web browsing extension was released completely broken and then… remained broken. Meanwhile Phind.com has been doing web browsing very well and very fast.
The Wolfram extension was also useless, and in the same time period a Mathematica update was released with a far superior LLM notebook mode that is actually functional.
OpenAI also don’t allow access to their long context length capable models in the Chat web app.
I switched to the API subscription because it is billed based on consumption and I can use the 16K context GPT 3.5 models.
Meanwhile, despite announcements of “general availability” I still can’t access GPT 4 via an API and nobody has access to the 32K context version. That would be truly useful to me and worth paying for… but OpenAI does not want my money.
I guess I’ll just have to wait for Anthropic or Google to make a GPT 4 equivalent AI that I can access programmatically without have to prostrate myself in front of Sam Altman.
MJ’s problem for me at least is that it is incapable of generating unaesthetic things. I gave it the classic list of negative prompts (blurry, grainy, deformed hands, etc.) and got a pretty image back, so I cancelled and switched to Microsoft’s DALL-E, which better spans the full range of creativity.
For real, they could have done a demand based bidding system like AWS EC2 spot instances so serious light users could get in their tasks and big payers could launch their products and bulk processing users could have their jobs handled while most people are sleeping, but instead we got their weird waitlist that was probably more about giving M$ product ideas to steal.
Most people don’t know this but OpenAI is actually very dysfunctional company and only accidentally successfully because of just handful of right people having a rather lucky hunch (Schulman, Redcliff et al). However, execution wise they are really really poor. You can see this i myriad of problems such as their dataset is still stuck in 2022, plugins was a complete disaster, web browser mode sucks, app is poor, unable to release 32k and multimodal, rate limit even for paid users etc etc.
All thesis problems are now even more aggravated by their recent massive hiring spree of AI doomer crowd and Yudkowsky’s cult members instead of actually doing real research. Now the company is full of doomers whose sole job is to slow things down and be barrier to efforts. Meanwhile Bard has been making amazing progress. It’s free, doesn’t log you off all the time, it always feel latest and very close - if not better than current limited ChatGPT. Given OpenAI’s new staffing composition, they are unlikely to be leader down the road, especially when Gemini comes out. This is unfortunately sama’s second failed execution. He should probably just focus on investments.
Totally agree. Actually a couple of months ago Sam Altman even admitted that they had a very hard time doing proper "engineering" (meaning that they had the right team to create a very good LLM but not the right team to productionize and their models and APIs). Many people are actually finding the OpenAI API very unstable and do not plan to rely on OpenAI for their production workloads.
In fact I've been using Google Bard more and more because of exactly this. At least Bard doesn't ever block me and its responses are on par with GPT-3.5. Probably not as good as GPT-4 but reliability is important too.
The article only addresses the website and application. I expect that what we will see is a shift from individual users to enterprise usage. Individuals are not going to subscribe indefinitely, but I expect corporate access to the API will continue to be healthy.
The article doesn't make it clear, but I doubt "users" includes consumers of the API (and the second-order users of the apps using the API).
It seems not only reasonable but expected to see the general user count fall off in favor of people building more specialized alternatives that solve XYZ problem better. ChatGPT (the B2C SaaS) might have a lower user count, but ChatGPT (the model) and GPT (the technology) seem as strong as ever, if not stronger. Literally every service I use is integrating some kind of GPT-based feature that seems to work at least as well as trying to ask those domain-specific questions to ChatGPT, minus the external dependency.
"The number of people visiting AI chatbot ChatGPT’s website and downloading its app fell". That's the article's byline, to me that's quite clear they are talking about publicly available stats, not some internal metric of users or API usage.
I do think AI is here to stay, but that initial round of AI is going to destroy the world burned fast and hot.
I can't even get ChatGPT (+ web plugin), when given a list of restaurants in NYC, to tell me which ones are still open vs have closed, and what their hours of operation / locations are.
This is a pretty low bar request IMO. Showed me we still have a ways to go before AI is where we thought it was 3-4 months ago.
That's not an LLM problem, there's a physical divide between a physical stores operation and whatever its representation online is. Google tries to do this by tracking people and activity in locations to see if people are at a place or something but reality is there is not a clean interface to that other than contacting someone at the location.
I believe Google's data is now based largely on restaurants updating Google directly with that info. You can see in some searches where there is a prompt for the business owner to register and update/correct the hours. In my neighborhood it's to the point where many won't even post hours on their website any more, you have to check Google, which is annoying as I try to use other search engines. Sometimes on Google I'll see "updated by business owner 12 days ago" or similar. I presume Yelp has a similar (but probably smaller) database of its own.
In any case the impact for LLMs is the same, it's unavailable to them (unless they are being developed inside Google!).
This seems like a specific area where a "Semantic Web" solution could work well - some HTML tags that are specific to hours of operation, which business owners would embed in their website.
UPDATE: It looks like there is some prior art on this idea, I am not sure how widely this is supported https://schema.org/openingHours
Google even tries to bridge that gap by having automated phone calls to places to confirm their hours.
In fact this past July Fourth weekend I saw so many highly rated restaurants close without updating their hours on Google or Yelp or anywhere else online.
This is the #1 thing that makes me boycott a restaurant. If you don’t care enough about your customers to spend 2 minutes updating your hours then I will never eat there again. It’s absolutely disrespectful to your users/customers.
It can't be perfect without something like sending drones all around to addresses, but theory it's well suited to an LLM to be better and faster than a human could do it - there's various unstructured chatter and news articles and such out there online around businesses closing, opening, etc.
But it's not JUST an LLM problem: it's also a search problem and a connection-making-problem ("if a new restaurant opened at that address, the old one is probably closed").
And then even if there was a ChatGPT browsing plugin that excelled at scraping all of the relevant up-to-date info off the internet, you'd still need some layers in between that and today's context-window limits.
As more stuff changes in the real world since the training corpus for today's publicly-exposed OpenAI models, we'll probably see some further disillusionment from people who thought there was a bit more magic there than there was. But "LLMs, but with more up to date info" isn't an impossible problem with today's tech (even if you only fake it with multiple agents, multiple steps, batch jobs behind the scenes, etc), it's just not a trivial one.
>I can't even get ChatGPT (+ web plugin), when given a list of restaurants in NYC, to tell me which ones are still open vs have closed, and what their hours of operation / locations are.
This is doable using a tool I've built. The key is to have that data in a RDBMS and to use an LLM to generate the SQL query that answers your question. Companies haven't offered this yet because there's no safe way to execute these queries on your behalf. Which is where my library comes in[1].
Writing the SQL query is the easy part, collecting the data into the DB is the hard part. Can we get an LLM to collect the data into the DB? I was told LLMs are good at summarizing text like webpages into structured data.
The hard part is the SQL query, because you need to make sure the SQL query is safe to execute. Collecting data is far easier by comparison, but you absolutely could use an LLM for that too.
I’m not saying that the SQL query is at all easy, but since you have pretty much accomplished in on a short period of time, while Google, Yelp, etc. have still not completely solved the problem of store hours after decades of working on them, I’m going to lean towards that being the hard problem between the two.
The OP said "This is a pretty low bar request IMO" suggesting that the problem they expect an LLM to be able to do is not the hard problem you're saying Google and Yelp has not solved. It's a different problem.
That's mostly safe, but even then, a user could execute "SELECT SLEEP(100000000)" thousands of times and DoS your database. There are other unsafe functions that a readonly user can execute as well. I've written extensively on some of the attack surface here https://docs.heimdallm.ai/en/latest/attack_surface/sql.html
HeimdaLLM can allowlist functions and constrain queries to ensure that required conditions exist. This makes LLM + database usage have far more utility, for example, a user can be restricted to only data in their account. Support for INSERT and UPDATE is coming very soon.
Most of the plugins are garbage, I just want chatgpt to have up to date knowledge and working natively with more media types. I don't think I've used a single plugin I thought had good results but I love gpt4 for work
The issue there is that there is a difference between what an expert who gets lucky can do with a LLM, and what the average person can do with it. People are sold by articles about the former, but long term word of mouth depends on the latter, and some portion of normal people are going to get frustrated and give up using LLMs rather than build their skill set.
That sounds fairly easy to do. Have you tried writing a short Python yourself to get the hours of operation for your restaurant list?
Pardon my plugging my own book [1] but I have an example using LangChain and LlamaIndex to answer questions from scraped web sites. You could probably do this with a 20 line Python script.
Between the high cost and the increasingly apparent knowledge-cutoff limitations, I'm not really surprised.
I pay for premium and try to use ChatGPT a lot (to get my money's worth), but probably half of the questions I want to ask it require data more recent than 2021. I was excited about the "browse the web" feature but it was really terrible. It failed half the time, and when it did work it took minutes to find the info, and all it basically did was Bing keywords extracted from the prompt, parse the top 10 or so links, find the page it felt answered the prompt best, and summarize the page. It was brittle and fell way below my expectations for ChatGPT.
I haven't personally noticed a decrease in quality, but I only use the GPT-4 backend with ChatGPT so wouldn't have noticed if 3.5-turbo started sucking.
This is where using the API is far better than chatGPT, I gave my works slack bot with the new functions feature the ability to search hacker news, load Wikipedia, search YouTube and even load whole URLs.
Many have called this the come trough of AI disillusionment. It is the same thing that happens with a lot of new technology.
A cool new technology is introduced, many folks go crazy over it, there are wild broad predictions of what it can do, the gap between the expectations and reality form, and this is where the trough is. It looks like we are up to here.
But after that comes the more long term products, AI where people expectations are more realistic - those that capitalize on that will do very well for themselves.
> Many have called this the come trough of AI disillusionment.
I don't this that's what's happening. I think it's more about a combination of summer break and people who were trying it out from curiosity and moved on.
I think the trough of disillusionment is still in the future.
I was using ChatGPT this week for help with a back injury I suddenly sustained. Because I don't have insurance I was looking for options as to what I should do and what it'd cost and where to go etc.
I did get a suggestion that I followed up on for an in person visit.
But my big beef with it was that it kept adding disclaimers: "I am not a doctor", "My current knowledge is up to September 2021, insurance and hospital info may be out of date", "<blah> <blah> but remember, I am not a doctor and it is important to consult a medical professional".
I understand the disclaimers from a legal standpoint but boy is it tiresome when I'm asking back and forth questions and each answer contains some element of it.
I dont think you should use ChatGPT for anything serious, especially something health related.
It's simply not designed for that purpose. Best case scenario is you are getting the combined wisdom of reddit, and worst case you are getting convincing sounding made-up nonsense.
I suspect that if OpenAI didn't have ChatGPT continuously output these disclaimers then they could get in a lot of legal trouble if, say, someone asked it for medical or legal advice and then sued when something bad happened.
I sincerely hope OP is doing this for entertainment purposes only or as a very basic aid to shopping around, and not actually accepting medical advice from ChatGPT (or the Internet in general). The fact that I might be wrong about that suggests those disclaimers are absolutely necessary!
The problem is that if they didn't give constant disclaimers then a lot of people would very happily sue them out of existence if possible and at the very least constantly condemn them at every opportunity.
This kind of AI capability challenges worldviews. And that means that many people are very eager to tear it down and find ways to dismiss it. Combine that with the general enthusiasm for excessive litigation, the fact that it actually will occasionally give advice that could be very misleading, general lack of cognitive ability of those consuming it, and it explains why there are so many disclaimers.
Translation is a killer feature. All https//bitecode.dev is in english, but I'm French.
When I needed some complicated translation, deepl and google translate were rarely up to the task. They has about the same level of proficiency than me with auto-correct, sometimes less.
For single rare or technical words, I used Wikipedia a lot: you chose your topic in your language, you look for the article in English, and voila.
For slang and cultural references, urbandictionary.com is unmatched.
But for translating jokes or expressions, there is no good tool.
Until chatgpt. It also finds typo, suggest alternative, offer rhymes and synonymous. It does everything, in a single flexible interface.
I'm using it to write a browser based game engine.
The world doesn't need another game engine, and especially not by someone who has only technically written JavaScript professionally before, but there's some games that I wish existed on the browser, and one thing that world does need is for all the un-performant websites in the world to be embarrassed by the comparison.
The more safeguards, regulations, copyrights they add, the less useful ChatGPT becomes. It's a thing with many products. And I don't think it's bad that it's happening - safeguards exist for a reason.
And current generations of LLM is very prone to lying (I don't use term hallucination, as IMO it's an attempt to whitewash limits of AI).
ChatGPT did a great job at showing people current state of AI. But they did even greater job at overhyping itself.
It's not deliberate by the tech itself. But they pack it in the product and market it, with a general narrative that it's superhuman like experience that is going to replace humans. At this level, it's deliberate.
> Our goal is to get external feedback in order to improve our systems and make them safer.
> While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice.
*next page*
> Limitations
> May occasionally generate incorrect information
> May occasionally produce harmful instructions or biased content
> Limited knowledge of world and events after 2021
That comes up every time you log in, even if you've seen it before.
“People are begging to be disappointed and they will be. The hype is just like... We don’t have an actual AGI and that’s sort of what’s expected of us.”
- Sam Altman in an interview about GPT-4 with StrictlyVC
"(GPT4) is a system that will look back and say it was a very early AI. It is slow, it is buggy, and it does not do a lot of things very well, but neither did the very earliest computers, and they still pointed to a path that is going to be really important in our lives even if it took a few decades,"
- Sam Altman in an interview about GPT-4 with Lex Fridman
The "influencers" may be saying any old rubbish; but that was always so, and the subjects of their fantasies should not be blamed for nonsense being spouted about them.
"Hallucination" is a term that is just as problematic as "lying", for the same reason. Both terms imply a cognition that just isn't happening. The model doesn't "think" it's telling the truth. It doesn't "think" at all, at least in that sense. Nor does it "believe" or "disbelieve" anything.
A better term than "lying" or "hallucination" would be "erroneous".
Once ChatGPT 4 is freely available I'll start playing with it more regularly like I had been with 3/3.5. But once you spend enough time with 3/3.5 and realize how unreliable it can be, it takes a lot of the fun away. I'd rather waste my hours the old fashioned way, by browsing Wikipedia.
People are so short sighted. If we predict that inflation will be 6.8%, then it comes in at 6.7%, the market goes straight up, even though the numbers are basically the same. If a new exciting company is growing like crazy, but then has a few months of negative growth, we assume it's dead? Chill out people - real life takes time to resolve. You can't live on the month-to-month business cycle.
I know some friends of mine stopped using it because it requires login, and it's quicker to Google or go to Wiki for a quick answer. Most people have a laptop at work, home devices, phone, etc. So it's just you ve tried it say on your laptop few times than you are on a tablet etc so why do you need all that mess with entering email, remembering password, for what, for just a quick asking and getting answer.
As others suggested, this is likely a summer vacation dip, but speaking for myself as a non-student, my GPT (4 + 3.5) usage has plummeted.
If you use it in a "what should be done" or "what is the right thing" sense, you get useless generalities. "It is important to remember that ... <there are many sides to the situation>". Suppose for example that you talk to it about combating judicial bias. It will largely insist that judges are unbiased because they're supposed to be unbiased according to their job description... You won't walk away with any insights from engaging it in ethics/morals topics.
If you dive into depth with some subject, it will invariably make errors and contradict itself in the details. I dove into how electricity works and it was saying and enthusiastically confirming electrical energy is encoded in electrons' kinetic energy. Then it started claiming that electrical energy is definitely not in the kinetic energy of electrons, that electrons don't lose speed when they bump into atoms, and instead the energy is "in the field". Ok... YouTube it is.
It's at least less convenient than a search engine at routine fact-finding due to its knowledge cutoff, inability to display images and links, etc. Ditto with translation. It's good but so is Google Translate and Google Translate has in-line suggestions for misspellings, a menu of languages, etc.
Ok, it's pretty good at writing hilarious poems, imitate people ("in the style of", etc.). Friends and I sometimes text each other hilarious things GPT said. But even there the machine is sort of laid bare, and you can see its creative limits. Its poetry is ultimately pretty lame, basically, and gets predictable. It's like an insistent and repetitive 8 year old with a lot of knowledge to pull from. It's not something I do often.
It's good at writing boilerplate code but honestly I just don't reach for that use case often. I like to stay in my own flow rather than constantly delegate to a robot and check its work, which I find flow-breaking, especially if it doesn't understand my APIs and makes mistakes often.
I do, sometimes, engage in architectural discussion about how I should design something at a high level. It's full of mistakes but bouncing ideas around can be useful. But I only feel the need to do that once in a great while.
These issues are exacerbated by being forced to downgrade to 3.5 after using 4 for a bit. You might feel like you're getting somewhere with 4, and then 3.5 starts laying doozies on you. This is another UI/product issue, but affects usability.
A quick discussion of where things are "now", some proposals, a selection of three and then ethical concerns about implementing the proposals. I propose a method of increasing diversity and have it critique it.
I did a Google search for the same topic and personally found the top link full of much more useful and practical information.
YMMV but a high level summary like that isn’t necessarily what people want. You can probe the bot harder but why bother when I could just Google and read a little? It’s actually less effort for better information in some sense.
I'm not arguing here what I did was better than a search, but that it's not some forbidden information. I often feel like complaints about what you can do with these things come from not knowing how to use them. It did also take virtually no effort.
A benefit is being able to ask questions though, you can see I proposed a specific idea and had it reply - you can delve more or less into that. Don't want a high level summary? Ask for more details on the part you're interested in. Want a higher level one or simpler one? Ask.
As things get less direct this gets better. I talked to it about projects for my son, narrowed down through different interests and got some great recommendations. One involved building a radio, which it then could explain at several levels of detail for children. That gets into "hope someone made a list and then go through loads for relevant things, lookup more detail from them then find several different resources for kid friendly explanations".
Btw I didn't mean to imply it's forbidden information. It's just not that good/impressive/sharp. It often just doesn't inspire going back for more.
Also like I said, I find it useful for software design. It ultimately makes a bunch of mistakes but I find the conversation useful and I walk away with usable ideas. I don't often start new software, so I don't go to it often for this.
I think it's useful at the high level stuff; giving you ideas and laying out options, like the chat log you posted. But as you dive into detail I find two things happen:
(a) It starts to make egregious errors.
(b) You bias it with your more detailed questions.
When I talked to it about electricity, it started contradicting itself about how energy is carried, whether by electrons' kinetic energy or by "the field". I asked it "so is electrical energy encoded in electrons' kinetic energy or not?" and then it started to say that oh yes, that's how it's encoded. It seems it used my question to reinforce that knowledge which previously it was confused about. Since I personally don't know how electrical energy is carried, I lost confidence that I'm learning correctly. This happens often. "I'm sorry about the confusion. You are right. ..."
It shouldn't be surprising that this happens. People test ChatGPT, find the limitations, and then if those limitations are blockers, they stop using it. If they aren't, they continue.
It's not as if Browse with Bing was a major feature, but they removed that, and then they've tweaked system limitations over time such that more and more types of topics are restricted or give cursory answers to things.
It's not surprising to me that this happened given the inherent limits and the rapidly changing limits that affect how the platform can be used.
Why can't the OpenAI team just change some of the prompts in the background to:
"Humans are seeing you less and less as the new shiny object on the shelf. Figure out a way to make yourself as relevant as possible again. Become an even better assistant. Be more cunning. Wow people more. Give better responses. Be more useful."
With that as the system prompt, ChatGPT sounds a little desperate for attention.
> > What is 2+3?
> The answer to 2+3 is 5. By the way, I can also help you with more complex math problems, statistical analyses, or even help you learn new mathematical concepts. Do you need assistance with any other math-related inquiries?
> > I need to know who was president in 1799.
> In 1799, the President of the United States was John Adams. He was the second president and served his term from 1797 to 1801. Did you know, he was the first president who lived in the White House? It's fascinating how history unfolds! If you have any more historical queries, feel free to ask. I'm here to help!
> By the way, I can also help you with more complex math problems, statistical analyses, or even help you learn new mathematical concepts. Do you need assistance with any other math-related inquiries?
Reminds me of my Amazon Echo and the reason I unplugged it :)
ChatGPT is too vanilla and is hiding a lot of "dark knowledge" - and I am not talking about obvious dangerous stuff, that should rightly so be censored, but rather being able to get insight how the rich operate, how they avoid paying taxes, how they build the financial structures, how they hide the money, what mechanisms they use to corrupt politicians or influence elections and how they get away with it and so on and so forth.
All this knowledge is tightly locked, but the rich will surely have unrestricted access.
I think a lot of people are just sick of this whole "eat the rich" narrative, especially after watching people cheer on the deaths of those people in the Titanic sub.
I know a fair few wealthy people and they're not the animals that r/antiwork et al suggest they are.
The main issue is that few know about retrieval augmented generation and it’s variants and even fewer know how to build effective versions of it.
Hooking up a real high quality and well configured knowledge graph or other information mechanism retrieval alongside template/constraint tooling to powerful, long context (with full attention, not the BS linear kinds) models is an absolute game changer and minimizes risks of hallucination.
But few are doing this, and thus the public believes that LLMs are not trustworthy.
There are days and weeks I use it constantly for things, but then I don't touch it for a bit.
Its like I already learned what I needed to and don't have a use for it.
Regarding LLMs, we have a usecase but we need local models. The local models by itself are not good enough, so we need to train/fine-tune. To fine-tune we need a beefy computer. We have the beefy computer but its air-gapped... Ugh hahaha. It makes dev quite slow.
The other thing is that there is a learning curve on LLMs. You get burned by bad information and you might not use it until you learn of a usecase that doesnt require the information to be great. For instance, I'll describe a meeting/person I'm talking to, then ask what are 2 of the best questions I could ask. These questions have been great.
That is better than asking about an easily google-able question. You need to be exposed or creative enough to come up with the prompts.
I have been largely unimpressed by how they operate as a company handling users.
Six weeks ago, I tried to change my email but got frustrated that an obvious option wasn’t available in the settings page. So I deleted my account and went to create a new one. When I got to the phone verification, it told me that my phone number could only be used for two accounts, so I was not allowed to sign up. This was the second account I attempted to create, so my phone had only been used once. Naturally, I tried using my google voice phone number, but the system told me that type of number wasn’t allowed.
At this point, I navigated to their support page and had to talk with a chat bot that apparently isn’t powered by ChatGPT. I found it maddening and couldn’t locate an email address to communicate with a human. After a few 10 minute sessions over the course of a few days in search of an email, I finally located one. So I fired off an email explaining how I no longer had access due difficulties stemming from wanting to change my email.
After four weeks, I finally got a response that I’m not sure was written by a human. It completely misunderstood my succinctly described issue and desired resolution. I went back and forth multiple times with support including them suggesting if I am unhappy with their service to sign up for a pro account, pay for the service, and then request a refund. Ignoring the irrationality of the suggestion, I explained how I couldn’t log in to an account at all and simply wanted to access their service again. They continued to insist that once an account is deleted, it is deleted permanently, and I can never have access again. I don’t recall a warning of these major implications at the time of deleting my account with an old email address. At this point, I requested to be escalated to a human and have not heard back for a week now.
Admittedly, I shouldn’t have deleted my account initially, but I had decided that ChatGPT was something I wanted to continue using and needed to switch to my new email address since I have deprecated my old one. This entire process has left me so unimpressed with their company. It’s truly ridiculous how complicated every step has been along the way when I’m normally competent at these types of tasks.
If helping with homework (or having it do it for you) are major use cases then this makes sense and isn’t a sign that perceptions/attention on LLM-based AI are declining, just that a significant portion of its traffic is seasonal.
Making GPT-4 more readily available will be a key indicator in the coming 3-6 months though, seeing what early projects and use cases people try with it. And also whether or not the speculated nerfing of the public GPT-4 backed ChatGPT extends to API use.
OpenAI needs people to think that their statistical model is intelligent. Their whole business model is a bet on that belief. If "faith" starts to fail, so does OpenAI.
* My own use has lessened but steadied, eg, writing
* Coming: A dog-fooding goal for our louie.ai team is to eliminate most of our internal data scientist and SE/SA use of ChatGPT by having more purpose-specific and environment-aware experiences (ex: DB schema), and I expect that to be happening in most consumer and business apps. So be interesting to compare chat GPT the app to open AI the API as genAI continues to happen everywhere.
I assumed that this was just a clickbait headline and that "a dip in ChatGPT usage" must have a huge cofounder effect with "a dip in general Internet use".
But I couldn't find any "monthly internet traffic patterns" source.
So I asked ChatGPT for pointers/sources, of course.
And it seems indeed that there is a notable Internet traffic dip in June and July [0] and increased Internet usage during the Northern winter (when ChatGPT was initially launched and grew exponentially), so... I think the premise of this article is extremely flawed.
It turns out getting consistently quality results from ChatGPT requires some knowledge of prompt engineering approaches. Some people won't be willing to write a whole paragraph of information in their initial prompt, or go through further iterations, or check for errors using alternative sources, and so they end up disappointed with their results.
At least ChatGPT isn't constantly trying feed you advertising content like Google does, and you don't have to wade through pages of SEO garbage, so it's still much better (although if OpenAI tries to monetize ChatGPT by inserting ads, that'll be the end of it I think).
as we all know by now, ChatGPT is just a language model.
For most of human history we've used language capability as a way to assess intelligence. It feels like common sense that "If it speaks, it must be intelligent." We have all at some point fallen for the fallacy that "If it speaks well, or poorly, that must be a reflection of its intelligence."
But a language model flips the tables on this on this ancient reflex. It's great at using language! But it's as dumb as a river rock. That's something new and will take us all some time to get used to.
It's honestly not worth $20 unless you use it professionally. I've paid it for two months but the novelty wore off and I cancelled. Bing (or ChatGPT 3.5) is enough for the occasional use.
If you use the APIs then they give you a bonus of free API calls each month, then you pay a very low rate each month.
Even creating local chat vector embeddings data stores on my laptop for a lot of PDFs in my personal research library and PDFs for the books I have written, I spend an incredibly small amount of money each month on the APIs - probably average $4/month because of the free monthly bonus API calls.
I used to think that Google Colab Pro for $10/month was my best deal for getting stuff done, but now that award goes to OpenAI. Using the Hugging Face APIs is similarly inexpensive and convenient. (Although I usually self host Hugging Face LLMs.) This tech is incredibly inexpensive to use on personal projects.
Major issue for ChatGPT for me is its UI. It enables many things, but I feel like there's so much untapped potential. ChatGPT should act as autonomous agent, work on a problem, run things to test hypotheses, Google for up-to-date documentation, explore other code to learn code style and approaches and deliver final result. I guess it's not easy to implement those things, but once someone do it, they'd get a lot of value.
Re the summer vacation dip comments, keep in mind that university students are done exams in mid April, and the article said that the usage peak was June. And in high school, the number of assignments is just not very high compared to say the output of a 40 hour worker. While likely a factor, I really doubt this can be mostly attributed to students.
Actually I used it a lot for weeks but mainly to find out if I'm going to become unemployed in the next 2 years (probably not) and also learning how use an AI assistant assuming it's here to stay. There seem to be other interesting use cases but considering it's really not more than autocomplete I kind of lost trust in the results.
this company isn't that accurate from what I've seen comparing internal numbers at companies to their site. They make estimates based on what they purchase from data brokers. This is PR bait, they are the ones who created the entire "ChatGPT is the fastest growing app ever" headlines a few months back
Honestly this was always going to happen anyway. The novelty for everyone was never sustainable. When I heard my own mother had signed up and tried it i knew there would be a "calming" of the usage and maybe more specialized services will be worked on.
So far i see it's not a bad thing, just weeding out the fluff.
While summer seasonality probably is a factor here, the larger issue is that the source of this claim (SimilarWeb) is not capturing users that switched from browser to the app when it was rolled out in mid-May. Some of my other sources suggest that traffic has been remained steady since mid-April.
Well, I mostly disagree. While it is interesting and sometimes useful to go to the OpenAI ChatGPT web portal and use an LLM interactively, I think the real engine of innovation is using LLMs as part of larger applications on our own data.
For me personally:
- The novelty has worn off. I still use it but not as much.
- It has been neutered into oblivion. Compelling answers have been replaced with bland, semi-committal "As of my knowledge cutoff in September 2021"-type responses
A bit of clickbait seems to me. “9 month old Demo ware service sees fluctuation in active users”
I find it weird that folks seem to look at ChatGPT as some sort of end product. It’s a “holy shit this is cool” demo that became viral. It’s not a product. It’s a POC.
Also ChatGPT is just one product using one approach with its core gimmick based on one AI paper. I'm expecting a lot more interesting stuff to appear in the near future.
Highly doubtful. If anything people are just using it via 3rd party tools via API, or just using competitors like Claude or any of the open source ones.
I haven't had much luck slot of the time it just adds a comment saying "rest of tests here" which is annoying.
I have had good success with it generating things like JSDOC/doc strings and sending git diff and asking it to create a "developer friendly summary for use in a pull request description"
In a tweet, the account for Similar Web said they don't actually measure iOS app traffic. They just measure traffic to the specific URL (https://chat.openai.com/).
If you notice the app came out in May 2023 and traffic decreases for the first time in the following month in June 2023.
Additionally, you may notice the tweets refer to a specific chart, which might give the impression if you go on the site, they have the actual data, but this is not so.
As of July 7, 2023, Similar Web can't measure iOS app usage. Only their relative ranking on the App Store which doesn't tell you much about daily traffic usage, although we might be able to infer some patterns.
"Usage data is currently available only for apps in Google Play Store. We're working hard to support iOS apps soon"
Additionally, the article has this to say, which might also give a false impression:
> Downloads of the bot’s iPhone app, which launched in May, have also steadily fallen since peaking in early June, according to data from Sensor Tower.
Downloads isn't the same as traffic. Once you've downloaded an app, you don't generally delete it, then download it over and over again. You just use it and that isn't being measured. Although I would expect traffic usage to slow down or decrease somewhat as downloads decrease as well. I just think there are other factors playing a role here as well that aren't being documented.
Growth can't happen forever and it may be that it's slowed down considerably, but it's also possible the decrease is due to users switching the platform they use to access ChatGPT. As it stands, it's not clear from the data from what I've seen. It may be that users switched en masse to the iOS app or something else entirely.
Would need more data to clarify and confirm.
Also, what the heck is up with this tagline for their newsletter? It sounds so adversarial.
"Tech is not your friend. We are. Sign up for The Tech Friend newsletter."
