A disposable sandbox wont protect you from secret exfiltration. Assuming you don't consider your code a secret, you could of course set up your sandbox so it doesn't have any secrets, but that would severely limit the kinds of tasks you can use the agent for.
On the one hand, you can set up a proxy that supplements secrets for API calls. On the other hand, you can whitelist what you need, in the simplest case with iptables (The devcontainer in the claude code repo is an example of the latter).
If flying ever becomes efficient energy-wise, this may happen. However, right now, flying is very energy inefficient, so anything that doesn't need to be flown, is transported overland to save costs. A change of fuel won't change it, unless the underlying energy usage changes fundamentally.
Better batteries do not impact energy usage, only the means of energy delivery.
The data centre runs on a dedicated power line. My laptop runs on battery. Using coding agents currently drains battery quite fast, which is surprising, given that the vast majority of the work does not take place on my laptop.
Making the client side coding agent more efficient isn't about saving the climate. It is about extending the workday (which might actually make the climate worse)
While i agree, the current JS security model rally doesn't allow for distinguishing origin for JS code. Should that ever change, advertisers will just require that you compile their library into the first party js code, negating any benefit from such a security model.
> advertisers will just require that you compile their library into the first party js code, negating any benefit from such a security model.
It will become harder for advertisers to deny responsibility for ads that violate their stated policies if they have to submit the ads ahead of time. Also site operators will need a certain level of technical competence to do this.
More likely, advertisers will need you to insert a "bootloader" that fetches their code and passes it to eval().
Alternatively, they might require you to set up a subdomain with a cname alias pointing to them (or a common CDN), negating any security benefits of such a practice.
> More likely, advertisers will need you to insert a “bootloader” that fetches their code and passes it to eval().
Sounds like legal precedent waiting to be set. “Run our code so that it looks like your code, acts like your code, and has all the same access as your code” seems like it should be a slam dunk if said code ends up doing a Very Bad Thing to your visitors.
But of course that’s assuming common sense, and the law’s relationship with that isn’t always particularly apparent.
There is already plenty of precedent for real-time-served ads which are annoying, or malicious, or install malware; or outright exploit vulnerabilities in the browser.
The advantage would be that I know beforehand, and have the opportunity to test and, possibly, reject, what the advertiser want me to send to someone’s browser.
Their own (presumably cherry picked) benchmarks put their models near the 'middle of the market' models (llama3 3b, qwen3 1.7b), not competing with claude, chatgtp, or gemini. These are not models you'd want to directly interact with. but these models can be very useful for things like classification or simple summarization or translation tasks.
These models quite impressive for their size: even an older raspberry pi would be able to handle these.
There's still a lots of use for this kind of model
Hi all, Ishaan from LiteLLM here (LiteLLM maintainer)
The compromised PyPI packages were litellm==1.82.7 and litellm==1.82.8. Those packages have now been removed from PyPI.
We have confirmed that the compromise originated from the Trivy dependency used in our CI/CD security scanning workflow.
All maintainer accounts have been rotated. The new maintainer accounts are @krrish-berri-2 and @ishaan-berri.
Customers running the official LiteLLM Proxy Docker image were not impacted. That deployment path pins dependencies in requirements.txt and does not rely on the compromised PyPI packages.
We are pausing new LiteLLM releases until we complete a broader supply-chain review and confirm the release path is safe.
From a customer exposure standpoint, the key distinction is deployment path. Customers running the standard LiteLLM Proxy Docker deployment path were not impacted by the compromised PyPI packages.
The primary risk is to any environment that installed the LiteLLM Python package directly from PyPI during the affected window, particularly versions 1.82.7 or 1.82.8. Any customer with an internal workflow that performs a direct or unpinned pip install litellm should review that path immediately.
We are actively investigating full scope and blast radius. Our immediate next steps include:
reviewing all BerriAI repositories for impact,
scanning CircleCI builds to understand blast radius and mitigate it,
hardening release and publishing controls, including maintainership and credential governance,
and strengthening our incident communication process for enterprise customers.
We have also engaged Google’s Mandiant security team and are actively working with them on the investigation and remediation.
Maybe the news has distorted a bit after crossing the Atlantic, but waren't there substantial outrages after the bits that couldn't be touched had in fact been touched?
It would be nice to aggregate all that and put it under a "profile". Kind of like facebook, but your entire profile feed is just the long list of court records, assholery and screw overs for other people. I actually saw a version that someone did for Jack (Twitter's ex founder) a few years ago and it was hilarious but cleverly informative. That's honestly where I got this idea from.
Two things are holding back current LLM-style AI of being of value here:
* Latency. LLM responses are measured in order of 1000s of milliseconds, where this project targets 10s of milliseconds, that's off by almost two orders of magnitute.
* Determinism. LLMs are inherently non-deterministic. Even with temperature=0, slight variations of the input lead to major changes in output. You really don't want your DB to be non-deterministic, ever.
From what I understand, in practice it often is true[1]:
Matrix multiplication should be “independent” along every element in the batch — neither the other elements in the batch nor how large the batch is should affect the computation results of a specific element in the batch. However, as we can observe empirically, this isn’t true.
In other words, the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies! This nondeterminism is not unique to GPUs — LLM inference endpoints served from CPUs or TPUs will also have this source of nondeterminism.
"But why aren’t LLM inference engines deterministic? One common hypothesis is that some combination of floating-point non-associativity and concurrent execution leads to nondeterminism based on which concurrent core finishes first."
reply