Too bad the book seems to be using Python and some external library like tiktokens just from chapter 2, meaning that it'll basically stop working next week or so, like everything Python, making the whole thing much harder to follow in the future.
Meanwhile i learned the basics of machine learning and (mainly) neural networks from a book written in 1997[0] - which i read last year[1]. It barely had any code and that code was written in C, meaning it'd still more or less work (though i didn't had to try it since the book descriptions were fine on their own).
Now, Python was supposedly designed to look kinda like pseudocode, so using it for a book could be fine, but at least it should avoid relying on external libraries that do not come with the language itself - and preferably stick to stuff that have equivalent to other languages too.
[1] which is why i make this comment (and to address the apparent downvotes): even if i get the book now i might end up reading it in 3-4 years. Stuff not working will be a major obstacle. If the book is good, it might end up been recommended by people 2-3 years from now and some people may end up getting it and/or reading it even later in time. So it is important for the book to be self-contained, at least when it comes to books that try to teach the theory/ideas behind things.
Myeah, C and C++ have the advantage that the compilers support compile for old versions of the language. The languages are in much flux partly because of security problems, partly because features are added from other languages.
That means that linking to external libraries using the older language version will fail unless you keep the old version around simply because the maintainer of the external library DID upgrade.
Python is not popular in ML because it is a great language but because of the ecosystem: numpy, pandas, pytorch and everything built on those allows you to do the higher level ML coding without having to reinvent efficient matrix operations for a given hardware infrastructure.
>Python is not popular in ML because it is a great language but because of the ecosystem: numpy, pandas, pytorch and everything built on those allows you to do the higher level ML coding without having to reinvent efficient matrix operations for a given hardware infrastructure.
Ecosystems don't poof into existence. There are reasons people chose to write those libraries, sometimes partly or wholly in other languages for python in the first place.
It's not like python was older than or a more prominent language than say C when those libraries began.
(i assume with "The languages are in much flux" you meant python and not c/c++ because these aren't in flux)
Yeah i get why Python is currently used[0] and for a theory-focused book Python would still work to outline the algorithms - worst case you boot up an old version of Python in Docker or a VM, but it'd still require using only what is available out of the box in Python. And depending on how the book is written, it may not even be necessary.
That said there are other alternatives nowadays and when trying to learn the theory you may not need to use the most efficient stuff. Using C, C++, Go, Java, C# or whatever other language with a decent backwards compatibility track record (so that it can work in 5-10 years) should be possible and all of these should have some small (if not necessarily uberefficient) library for the calculations you may want to do that you can distribute alongside the book for those who want to try the code out.
[0] even if i wish people would stick on using it only for the testing/experimentation phase and move to something more robust and future proof for stuff meant to be used by others
"The languages are in much flux" you meant python and not c/c++ because these aren't in flux
No I meant C++.
2011 14882:2011[44] C++11
2014 14882:2014[45] C++14
2017 14882:2017[46] C++17
2020 14882:2020[47] C++20
2024 14882:2024[17] C++23
That is 4 major language changes in 10 years.
As a S/W manager in an enterprise context having to coordinate upgrades of multi-million LOC codebases for mandated security compliance, C++ is not the silver bullet in handling the version problem that exists in every eco system.
As said, the compilers/linkers allow you to run in compatibility mode so as long as you don't care about the new features (and the company you work for doesn't) then, yes, C/C++ is easier for managing legacy code.
These are new features. Many of them are part of the library not the language. Generally speaking what you do is enable the new features in your compiler, you don't need to disable that to compile old code. It's not a problem to work on legacy code and use new features for new code either.
C++ has issues with becoming more bloated and complicated but it does not have issues with having existing code breaking due to these changes (aside perhaps some very edgy edge cases).
> That means that linking to external libraries using the older language version will fail unless you keep the old version around simply because the maintainer of the external library DID upgrade.
This just isn’t true. C ABIs has not seen any change with the updated standards and while C++ doesn’t have a stable ABI boundary you shouldn’t have any problem calling older binary interfaces from new code (or new binary interfaces from old code provided you’re not using some new types that just aren’t available). That’s because the standard library authors themselves do strive to guarantee ABI comparability (or at least libc++ and stdlibc++ - I’m not as confident about MSVC but I have to believe this is generally true there too). Indeed the last ABI breakage in c++ was on Linux in C++11 15 years ago because of changes to std::string.
These do not solve fundamental issues as mentioned elsewhere.
Hell, just a few weeks ago i wanted to try out some tool someone made in Python (available as just the Python code specific to that project off github) - but that was broken because there was some minor compatibility breakage between the version the author used and the current one. The breakage wasn't in the tool itself but in some dependency 3-4 layers down (i.e. some dependency of a dependency of a dependency). Meanwhile the exact version of Python used wasn't available in the distro packages.
Eventually i solved it by downloading the exact Python version source code inside a minimal Debian docker container, compiling it and installing the rest via pip (meaning the only thing i changed was the Python version, not anything else).
Neither conda nor venv really solve the underlying problem.
Most ml project authors only supply a "top-level" requirements file, that is, one that only contains the list of dependencies directly required by the project, not a full graph of both direct and indirect dependencies.
Those dependencies are often unversioned. This means pip will fetch their latest versions, which might be incompatible with what the project actually requires, e.g. numpy 2 instead of the numpy 1.x that the project was actually developed with.
Even if the dependencies are versioned correctly, it's likely that one of those dependencies has its own improperly versioned dependency. Maybe the project doesn't depend on numpy directly, but it depends on foo 0.8.14, which depends on bar 3.11.4, which is sloppy and depends on improperly numpy, which will resolve to 2.x nowadays even though it actually needs 1.x.
You can `pip freeze`, but that doesn't always work, as the graph of required dependencies might differ across platforms and Python versions. It also has the problem of not distinguishing between the actual project constraints and those mechanistically added by pip.
Then there's the fact that pip installs dependencies one-by-one instead of looking at the whole list and figuring out what versions will work together. If you need both foo and bar, where foo needs numpy 1.x and the latest version of bar needs >2.0, you will get >2.0 and a broken foo, instead of a working foo an an older working bar that still worked with numpy 1. This is actually one problem that conda solves, at a serious performance cost.
Not to mention the fact that requirements files often fall "out of sync" with what is actually there. It is too easy to follow the suggestion in some exception and just `pip install` a missing package, forgetting to add it to requirements.txt.
Then there's the python version debacle, people often don't specify the version of Python needed, which sometimes just makes the requirements uninstallable (or, even worse, subtly broken).
The solution to all these problems is using uv. It keeps a package lock of the exact set of working versions (which is always resolved for multiple platforms), and which is separate from the version constraints imposed by the project author. It resolves packages all at once without Conda's performance problems, Rust probably helps here. It encourages commands like `uv add`, which install a package and track it as a dependency in a single step. It even tracks Python versions, fetching the exact version needed when necessary.
Those are decent options but you can still run into really ugly issues if you try to go back too far in time. An example I ran into in the last year or two was a Python library that linked against the system OpenSSL. A chain of dependencies ultimately required a super old version of this library and it failed to compile against the current system OpenSSL. Had to use virtualenv inside a Docker container that was based on either Ubuntu 18.04 or 20.04 to get it all to work.
To a certain extent. The big difference being how difficult it would be to work around or update. If my C code is linking against a library whose API has changed, I can update my code to use the new API so that it builds. In the case I ran into… it wasn’t one of my dependencies that failed to build, it was a dependency’s dependency’s dependency with no clear road out of the mess.
I could have done a whole series of pip install attempts to try to figure out the minimal version bump on my direct dependency that would use a version of the sub-dep that would compile and then adapt my Python code to use that version instead of the version it was pinned to originally.
Now if I only could find some time...