For anyone who would like to know more about designing proteins with a certain function, target, or structure in mind, the term to search for is "rational design."
As an aside, learning the precise terms for concepts in fields in which I'm a layperson (or simply have some cobwebs to shake loose)--and then exploring those terms more--is something that I've found LLMs extraordinarily useful for.
Technically apple-m4 is ARMv9.2a, but a quirk of LLVM defines v9.0 as requiring SVE, which is optional according to the Arm ARM and not supported by the core. ARMv8.7a is the next closest choice.
Hmm I wonder how this happened? I assume at least at one point SVE was such headline feature of 9.0 that LLVM assumed any 9.0 chip would have SVE even if it wasn't technically required?
Background: I taught high school science for half a decade and now I am in tech.
This will never take off. One, there is no money in Ed.Tech. There is no money in Ed.Tech. There is no money in Ed.Tech. What little money there is goes to the obvious stuff like student records databases. Anything that requires an ongoing subscription fee is dead in the water. The only reason those stupid smartboards took off is because they make school boards look cool, they are a one time cost, and can be paid for with bonds (because they are a one time cost). Teachers don't want them (projectors and document cameras are good, though). Ed.Tech is a wasteland of failed startups. Part of the problem is also that classic "the people with the purchasing power are not the people who will be using the product" problem.
Two, everyone outside of education thinks "well has anyone just tried sitting down with the kids and talking to them/explaining it to them?" Yes, obviously. The problem isn't that they are lazy, snot-nosed kids (that's a problem well within an experienced teacher's skill set to solve). The problem is what is the AI going to do with the kid says "fuck you" to the AI because they haven't eaten since lunch the previous day (school is the only place they get regular meals), or they don't even know what to ask because they are basically 4 grades behind in math, or the wifi is dead for the 8th time that month because the school board will never pay for infrastructure.
Three, what if the AI is just wrong and starts confusing the student? Even GPT-4 fabricates things all the time. Sure it can generally put words in grammatically correct order and is passible for writing no one is going to read anyway (like marketing emails). But the moment it requires actual domain knowledge all these AI models completely fall down because, again, they don't actually understand anything, they just are really good at guessing what word comes next.
I understand the skepticism around AI in ed tech, and I think people have the right to be skeptical of this being portrayed as a "cure all". Saying that it is of no value because nobody will pay for it and the kids wont learn from it because they're hungry does not capture the whole picture. I was never a great student because I had trouble focusing in class, but if I had this to guide me through my homework, I believe I would have been a much better student. Looking at math homework and having no idea how to even start and no resources to help is very different than looking at the problem and working with AI to help you understand how to attack the problem. Sure, you can't turn every single kid into a math wiz, but I think there is a real possibility that this will help almost every kid become better at math than they would have. Assuming this is low cost, I think many parents would be very willing to pay for it.
If you have ever worked with children you will realize vast majority of them lack the willpower to learn new things and seek out answers. A good teacher can provide the social accountability and guide them, but it's not something you can put on auto pilot with AI.
Children look to adults as role models for what to learn and why. We already know that children respond better to role models who have similar ethnicities and backgrounds to them, let alone being the same species.
AI cannot and will never be able to provide this motivation to learn, which is what kids actually need, because it is not human.
> AI cannot and will never be able to provide this motivation to learn, which is what kids actually need, because it is not human.
We hear all the time about digital addiction, gamification, FB, Tiktok, etc. Addiction is arguably more difficult to achieve than motivation. AI will be able to motivate people just fine
Using modern digital gamification to reinforce learning outcomes, a-la TikTok , will probably produce a generation of absolutely brilliant, weird psychopaths.
At the same time, I think asking a person or a parent requires much more will power than asking a computer (for whatever reason -- but part of it is just that a teacher might not have as much time as a child needs/wants). I do agree the social accountability of a good one-on-one teacher is the most ideal -- for me I got that from my parents/siblings. But lots of folks don't have access to that, and school systems don't have the resources to supply that, so maybe AI might be a good middle ground.
I am about a month away from finishing my teaching degree (math). My experience so far AND the best research is extremely clear on the following:
Kids learn best by working in small groups with other kids.
These groups need to be gently guided by adults, but they should mostly be left to do a lot of independent exploration and discussion amongst themselves. The teacher is there to prod discussion in productive directions, provide feedback, answer questions, give hints and encouragement where needed. Admittedly, AI could do certain part of the teachers job, but it can never replace a peer group.
There may be "no money in EdTech" but that doesn't mean there can't be impactful products for it.
Students don't want to pay for Google but every student still uses it.
Some students will want to waste time if given access to more tools like AI agents, but the motivated ones will use it to their advantage. Even if it's wrong sometimes, it's right a lot, perfect for rubber ducking, and almost always gives answers that are at least interesting or point you in a useful direction.
As for "GPT-4" fabricating things, I remember in school how I wrote essays and was given A's despite citing NYT and Wikipedia. Fabrications exist all over the place. If anything, hallucinations might give more examples as to how to be skeptical of any source and look for direct sources as much as possible.
In college I went to office hours and there was 30 minute line to talk to the TAs. AI agents respond instantly. These things are going to overall transform education for the better.
As for "GPT-4" fabricating things, I remember in school how I wrote essays and was given A's despite citing NYT and Wikipedia. Fabrications exist all over the place. If anything, hallucinations might give more examples as to how to be skeptical of any source and look for direct sources as much as possible.
"Actually it's good that AI sucks because it will teach people to be more skeptical" is not a very compelling argument.
Unfortunately, I think it will be true. Unfortunate, because we lack trust already in society and people can now revert to believing what they want to believe. But, they will be skeptical of everything else they read.
> As for "GPT-4" fabricating things, I remember in school how I wrote essays and was given A's despite citing NYT and Wikipedia. Fabrications exist all over the place. If anything, hallucinations might give more examples as to how to be skeptical of any source and look for direct sources as much as possible.
At least you wrote it by yourself? Copy-paste from GPT-4 removes the whole thinking process and need for basic skills. It is extremely difficult to prove that someone just copied the text without doing any own work.
If it is difficult to detect and punish, then certainly almost everyone will do that, if they get an advantage.
What is the future of proving that I know something myself?
>What is the future of proving that I know something myself?
What is the point of knowing something yourself?
This is just a reheated version of the argument against literacy by Socrates:
>You have not discovered a potion for remembering, but for reminding; you provide your students with the appearance of wisdom, not with its reality. Your invention will enable them to hear many things without being properly taught, and they will imagine that they have come to know much while for the most part they will know nothing.
You ask a very interesting question, that to me speaks to the future of education gets to the bottom of education entirely.
Right now, education focuses on getting people through the system so they can attain a credential. It teaches us, primarily, how to get one over on the system, how to look for loopholes and shortcuts. It creates a culture of corruption, of looking for approval from institutions in order to move up in life.
What does a world where anyone can learn anything they like for free very easily look like? Well undoubtedly it looks like a place where a lot of people don't learn much, because they have no interest in learning. But it also looks like a place where people who are interested in learning can and do learn endlessly if they like, the opportunity for anyone to get educated to a degree reserved for elites that most of us could only dream of even two decades ago is upon us. Writing the paper won't be about proving you wrote the paper to some authority figure, because it's no longer proof that you wrote it. Writing the paper will become about helping yourself understand something you want to understand.
So to your question, what is the future of proving that I know something, the future is you can't prove you know something, there's nobody to prove it to and you know things because you want to know things, because you recognize the value on knowing them beyond "this piece of paper qualifies me for a job." The future is the truth that we have been pretending isn't true for a century: that no matter what you try to do, those that want to learn will learn and those that don't won't, and their lives will be what they are on their own merit and of their own volition.
If you want to call yourself educated, the only person you need to be proving that to is yourself. We have entered a world of information post scarcity, which necessarily comes with post scarcity of noise, distraction and disinformation. The competition to understand things and the benefit that comes with understanding things will get stiffer, but access to that competition will become egalitarian. There will be a great divide among people: those that follow the noise of the day and those that continue to learn freely of their own volition.
>Right now, education focuses on getting people through the system so they can attain a credential. It teaches us, primarily, how to get one over on the system, how to look for loopholes and shortcuts. It creates a culture of corruption, of looking for approval from institutions in order to move up in life.
Maybe no school board buys and puts it into every class. Parents will still buy it and it can supplement learning gaps when doing homework. $20/month is a rounding error compared to pre-school, clothes, extra-curricular activies, private school, private tutors, etc.
The kids of people who can afford all you mentioned usually don't have problems with education. It's those who don't have $20.
My parents spent almost nothing on my education, just a couple of $ per year to buy a notebook or two. I used the same pens for almost ten years. I participated in zero sports, had zero tutors... I did ok but I had peers with even less resources and not all of them did good.
Fond memories of being mocked for wearing the same clothes I wore the previous year or two. Wait no, not fond.
The clothes were fine: good condition, clean, etc. Just old from the perspective of kids who didn't understand they came from money or that their parents were bad with money and headed toward disaster.
/u/raisedbyninjas was hopefully kinder than those kids even though it sounds like they were in the same cohort by how small $20/month sounds to them.
Given that Sal has never turned Khan Academy into a commercial product -- despite its resounding success and popularity, and the large amount of $$$ he could have made -- has proven he's not in this for the money. So I expect the AI version to also be free or at least have a free version for parents (and schools who can't afford it).
> The kids of people who can afford all you mentioned usually don't have problems with education.
Elite parents (read: the ones with discretionary money for schooling) do not see education in this way. They see it as an endless arms race to optimize every possible advantage they can for their child. From the moment the child was born, everything about their life was structured to optimize future earnings, which requires admission to an elite school, which requires exceptional learning tools.
The very people you are talking about that "don't have problems" are spending money and effort to get their kids to do even better, get into better schools etc. It's not some binary fine/not fine thing. They view it extremely differently to how you appear to be viewing it.
They won’t actively cut teachers, they just won’t hire as many specialists. We have a couple math specialists who just do remedial math. Kids who are at grade level or above get zero attention during math time. This will be a cheap way to provide attention to those kids, at a fraction of the cost of hiring more math specialists or classroom teachers.
Let’s say you want to purchase a math tutoring platform. The cost is $30 per student per year. CA schools get about $5 per student per year in lottery funds to purchase instructional materials (shared across all materials). General fund is pretty much all spent on staff/teachers or facilities. One time covid funds are running out. Schools can purchase textbooks at ~$100 per unit and use them for years without subscriptions.
In short, there is no money in public education for these tools.
These platforms look great, but the price is prohibitive. Maybe some basic aid district could pick them up, but those are usually small (I.e., it won’t keep a start up in business).
Note. This is an observation from California. Other states might find education better.
There is a small fortune spent on private education. After school classes, tutoring etc etc. The money is spent by the upper middle class and above. If this solution works better for some subset of those people, it will take off in that market.
I spend on private 1-1 sessions, group classes, books, apps, faceless saas subscriptions, toys, puzzles.
You misunderstand the buyers in this market. In my town, I'm in a good sized minority spending up on this stuff, despite being otherwise careful with money.
There is, but you usually have to rely on the gov't being the buyer, and not regular customers. I worked for a EdTech startup (catered to universities) so all our contracts were multi-year affairs paid basically by the government with the unis as an intermediary.
But also keep in mind that the people even getting involved in EdTech generally don't do it for the money, as is the case with Khan. The money my old company was making was pretty crazy, but basically all of it went to various funds and charities and what-not relating to education, because the CEO didn't care about making a buck and likewise with anyone working there.
I wholeheartedly agree with your points about AI though, outside of generating generic quiz questions and stuff like that I think it's a massive net negative on the entire educational system, though I suppose the cat is out of the bag now.
Basically, do you want a pipeline/process that utilizes LLMs and provides exactly the output you want? How do you formalize a human expert workflow using LLMs? With LLMOps. It is new, there is no standard or predefined resource that I'm aware of. At neuralnetes we are creating all of our methods ad hoc and use tight feedback loops to ensure ideal, precise output.
No need to set impossible standards here, the correct thing to ask is whether the guardrailed LLM confabulates less than its competitors. That sounds like a useful job to have.
I agree it's not a silver bullet doesn't begin to address the issue where students aren't learning because of issues they're dealing with outside of school.
I see this as Khan Academy x10. It'll be a great tool for those for whom Khan Academy itself was a good tool. If it's free (or for a nominal free), then parents will use it at home with their kids as a supplement to school. Teachers might use it on a case by case basis with students who want to accelerate or who are behind. It will highly accelerate learning for those who want to and are able to learn. The problems you describe, which are very real and important, are social problems, not educational problems per se or problems solved with technology.
First, clearly you had a crappy experience. And that sucks. There are probably many others with bad stories as well. I'm deeply involved with my kids education and can only imagine how much that sucked.
Second, we can't solve everything at once. Hungry kids, bad parenting, all that will take time, and may never be resolved. I wish it could be, but honestly, it will take time - probably a long, long time.
Third, he's building for the future. Richer schools (both private and public) will get this first, typical schools will get it next, and poor schools (both administratively poor and monetarily poor) will get it eventually. This is where that great quote applies - "The future is here, it's just not evenly distributed" - and boy will it apply to education. So much so that some parents will feel like they are left in the stone age by comparison.
Fourth, an anecdote. I do volunteer work for an education-related non profit. I learned during covid that HISD (Houston school district) couldn't easily move to online learning because 30% of their students did not have internet access at home. Thirty percent!!! I was blown away. Still am. But it's gotten much better, and we've evolved. Does it still suck? Yes, undoubtedly (HISD was taken over by the state, if that tells you how big their problems are). So I know this will not be evenly distributed because of this experience.
Maybe my reading of the post is incorrect but I did not interpret it as solving the parenting problem. In the US there is a massive growing problem where parents are simply not parenting their kids and relying on institutions (schools) to do it. I honestly do not know what will solve that problem.
To me this post shares a future where kids coming from good households will have the ability to learn more efficiently than before. It is not to replace the teacher but perhaps it will replace the tutor.
Lastly, I am not sure why but I genuinely believe that folks focus too much on "hallucinations" and "fabrications" by LLM. Does it happen, yes but I think its one of those things you can actually tune for and it get down to low to no errors in a walled environment. It is definitely something to consider but not a blocker.
> Sure it can generally put words in grammatically correct order and is passible for writing no one is going to read anyway (like marketing emails). But the moment it requires actual domain knowledge all these AI models completely fall down because, again, they don't actually understand anything, they just are really good at guessing what word comes next.
Many of us are frequently using it to obtain answers requiring domain knowledge; the evidence that this works is that the answers it gives are often found to be correct, and yet things we did not know. So your description is simply contradicting the experience of many people. Perhaps you're using it in a different domain, or not writing clear prompts, but it's well worth experiencing what people are talking about.
I can ask ChatGPT to write code and assess myself whether it's correct or not. (It's frequently not, but it can still be useful for certain tasks.) I can ask it to explain something about the industry I work in or adjacent fields and have a pretty good sense of whether it's leading me in the right direction. But if I asked it to teach me about, say, Chinese history, I would have very little ability to assess whether it's telling me the truth or sprinkling little falsehoods into things. I would only be able to catch the most blatant mistakes; the rest would require more detailed independent research.
Now imagine you're a kid who's still learning reading comprehension and hasn't yet developed a functional BS detector. You're not going to have much of an ability to separate truth from fiction, but you're likely going to trust this thing that sounds authoritative.
Yes, that's fair, but I think we can do better in our education revolution than just plonking kids down in a classroom with ChatGPT! If we take a fairly constrained subject like high school algebra, we should be able to, for example, study its BS rate by having knowledgeable testers converse with it at length, and thus perhaps produce AIs suitably calibrated and QA'd for classroom usage. But I should go read Sal's book, it sounds promising.
While these are all valid concerns, they also apply to human teachers. Humans spew bullshit all the time.
But that's OK. A teacher's knowledge doesn't need to be perfect, it just needs to be better than the student's.
If ChatGPT explains Chinese history in a way that is 80% accurate, to a student whose understanding of Chinese history is 10% accurate, the student will walk away with a better understanding of Chinese history.
This seems very incorrect to me. Human teachers make mistakes, or understand details of something incorrectly sometimes, but it doesn't come up in every lesson. It only happens occasionally, and sometimes a student will correct them/challenge them on it.
ChatGPT makes mistakes literally every time I use it (in a domain that I'm knowledgable in). How is that the same thing? Being given incorrect information is worse than not having the knowledge at all IMO.
I do find that current LLMs are quite bad at design problems and answering very specific questions for which they may lack sufficient training data. I like them for general Q&A though.
A different architecture or an additional component might be needed for them to generalize better for out-of-training-distribution questions.
“Fall down” doesn’t mean they’re always wrong. It means they’re right just often enough to make you believe them when they’re making stuff up, which is a terrible characteristic for a teacher.
At this point, ChatGPT is good enough that I can tell what authoritative source I should go look at for the answer, which is great, but I’m not going to believe any statements it makes without checking. I think people suffer a bit of Gell-Mann amnesia when dealing with ChatGPT.
I actually do get a lot of value from ChatGPT as a self-motivated learner with a specific goal in mind, but that’s not the scenario we’re talking about.
That’s a very limited perspective. While bulk of district spend (US) is salaries, buildings, pensions, busses, cafeterias etc. there is still a substantial spend on educational materials. It’s a tiny % of the overall budget, but the larger school districts have budgets in many _billions_.
Smartboards and 3d printers etc. got popular because they came from a DIFFERENT budget - these were capital expenses.
My personal experience (ed-tech reaching ~15% of school districts) is that it is impossible to sell a solution to a district _without_ support of the teaching and learning team - i.e. teachers. And if you do, it’s impossible to get a renewal.
Lastly, 9 out of 10 startups fail. It’s not just ed-tech.
There's some money but there's really two markets: small medium (eg a lot of work for a relatively small sale) and mega districts (and those generally just goes to the big established players because of connections)
Most school wifi isn't that bad anymore - the bigger problem for us is web filters that break things in interesting ways.
This might be the exception that proves the rule but I used to work for an outfit that made a school website product (SaaS) and they seemed to be making plenty of money.
That's more of a service for the school as an institution. I believe the OP is saying that most ed-tech startups that have a goal of helping with the learning process have not been successful.
The thing that comes to mind to me was all the hype around MOOCs and how much everyone thought they would be a game-changer for education, but the reality is that the problem wasn't the fact that people didn't have access to the education, it's that so few have the focus/discipline/resources to actually follow-through.
I'm willing to bet that it will be in the future. Maybe a free version that can run locally on Copilot+ level NPU's or GPU's, perhaps taking advantage of software & hardware cost reductions to get it down to a level that they can talk Gates into paying for it.
> I'm willing to bet that [Khanmigo] will be [free] in the future.
If anyone doubts this, just take a look at what Khan Academy currently offers for a fresh perspective. Make an account and click around (https://www.khanacademy.org/signup); it should be easy to see that a ton of effort went into making this and, as far as I can tell, they run purely on donations. I wouldn't be surprised if the current cost of operating Khanmigo is just being passed along by Khan Academy at no markup, or even a slight loss.
Who are not the learner which is the context of this particular writeup. Gates is talking about using LLMs to assist students as tutors, and that is not offered for free ($4/month, $44/year).
The idea is that teachers can use it with students.
For parents who want to use it with their children, $4/month (for your entire family!) is very small compared to anything out there that you want to use to supplement your kids education (other than Khan Academy itself, which is free). Is it affordable to poor families in Bangladesh? No, but the barrier to sponsorship is low: "for $50K a year you could sponsor it for 1000 families in Bangladesh" is not a difficult sell.
Also, I expect Sal to raise funding to give it for free to those who can't afford it (just as he did with Khan Academy). At least it's not some VC-backed venture that we know is going to be look for a 10~100x return, so we can be thankful for that.
Yes, teachers can use it with students if they pay for it. One day it might be free for students, but it presently is not. Which is what I was pointing out to the person I originally replied to. This discussion is about Khanmigo, a subset of what Khan Academy provides, and it is not free for students (though, yes, that cost is not exorbitant). But it does make it difficult to claim that Khan Academy is free when the specific feature being discussed is not.
I guess we'll see what the pricing is for school districts, but I expect it to be much lower than most educational tools, again given that it's made by Khan Academy which has proven that it's not out for the money. But something's got to cover the cost of the servers--maybe school districts will find sponsors. Overall, even if only parents and teachers use it, I'd say it's a big step forward.
Textbooks do a better job than 90% of teachers, and that's not a dig at teachers. I spend more time, by far, figuring out how to keep student attention than thinking about the material itself.
I'd be way more excited about getting 6 of my students to start reading and understanding textbooks than getting all 60 to pass. OTOH, my bosses would be pretty pissed...
IIRC studies show one-on-one attention is worth 2 standard deviations in test scores and a good teacher is worth 1 standard deviation. (Not coincidentally home schoolers have about one standard deviation of advantage).
Khanmigo doesn't have to be as good as a teacher to provide an advantage to the student.
> (Not coincidentally home schoolers have about one standard deviation of advantage).
Citation required as this is heavily dependent upon the purpose of the home schooling and the resources of the parents.
Anecdata: My cousin was not doing well in 4th grade. So, they pulled her out and her grandmother taught her for 8 hours a day, 5 days a week, every single day.
By the time she came back to school in 5th grade she was doing extremely well and hyper motivated to never go through that experience again. :)
She was, however, the exception, not the rule. So much so that a lot of teachers complimented her and her family on the outcome. Her family had no agenda other than "fix her education". Most homeschoolers do not bring to bear that amount of resource and attention, and they have quite a bit poorer results.
> AI does a better job than prob 90% of the teachers.
AI is better funded than probably 99.9% of teachers, so ...
Ahem. Go sit in a class sometime before spouting that kind of unsubstantiated claptrap.
Like anything with a distribution, some teachers are subpar. However, 90+% of teachers are trying very hard. They have too many students. They have too few resources. Many of them are fight against bad home lives of the students. And, through it all, they have a bunch of bureaucrats promulgating the latest and greatest bullshit that never works.
We know what works. Small classes (<15 students) with two teachers. This has been documented over and over and over. Nobody wants to pay for it.
My daughter was in highly rated school in NoVa some teachers would pretty much just put on youtube and be done with it. Now in CA again highly rated school just really sad especially compared to the school she was in Ireland. I am not even focusing on teachers that actually don't know the subject they are teaching. I mean teachers that will go into conspiracy theories or just blatantly say I am not in the mood so just basically do whatever.
> Anything that requires an ongoing subscription fee is dead in the water. The only reason those stupid smartboards took off is because they make school boards look cool, they are a one time cost, and can be paid for with bonds (because they are a one time cost).
Bonds take a one-time cost and turn it into a recurring cost (like a subscription) that is paid in installments over time. It's baffling that that it's easier to do that than just having a subscription in the first place.
The closest thing I have seen to this (which isn't that close) is the UI debug viewer in Xcode. You can get an exploded-view diagram of all the UI elements that you can rotate in 3D space. No lighting or shadows though and its limited to apps you have debug access to.
I think the Amazon Fire phone also tried something similar in real time with several front facing eye tracking cameras and the gyro+accelerometer to shift the phone UI and simulate a 3D view with parallax. The old mobile Safari tab view also used to shift the tabs based on the phone's orientation.
I would love to see a "globally illuminated" UI someday, even if it's impractical. Something like all those Windows renders the Microsoft Design team put out, but in real time. Totally impractical and a poor use of electricity, but it would be cool to have path traced, soft drop-shadows.
>I would love to see a "globally illuminated" UI someday, even if it's impractical
Apple patented a ton of stuff for this probably a decade ago. It seemed at some point they were going to start procedurally rendering aqua materials and the like using recipes for lighting that could all be dynamic.
It's similar to the "shipping your org chart" problem. Constant redesigns means you are shipping your employee performance review and incentive structure.
Nah Apple maps still sucks where I live. Lots of small streets and alleys that are on Google maps are absent on Apple's. Locations are slow to be updated. Driving directions that are illogical to locals. Maybe it's better in Cupertino/New York, but not where I live.
https://en.m.wikipedia.org/wiki/Rational_design