They have to use GCP as well, which is arguably a strong indictment of their experience with AWS. Coincidentally, this aligns with my experience trying to train on AWS.
"Remote material cooperation with evil" at best, which implicates virtually all human action. There is nothing immoral here. It's direct or formal cooperation that you need to worry about.
I don't disagree with the gist of their argument, but the fact that they try to whitewash an actual genocide [1] with "politics" is absurd.
1 - if anyone is confused, the UN convention on genocide explicitly lists taking children of an ethnic group to give them to another in the definition of genocide. Russia is quite openly and blatantly doing this.
My tax dollars are already funding genocide, so for like 10 cents to go towards Yandex a month, of which some fraction goes towards Russia's quixotic war effort (which is an international crime but not a genocide in intent or effect), is not something that's gonna keep me up at night. Almost every other purchase I make comes with harm roughly commensurate with that of Kagi. The damage to the environment Perplexity and Google (and Kagi) cause with unnecessary AI usage is a much bigger concern to me personally.
> which is an international crime but not a genocide in intent or effect)
Why not? Russia has kidnapped hundreds of thousands of children, gives them for adoption to Russians, and claims that Ukrainians are just confused Russians.
If it smells like a genocide, fits the definition of genocide... it's a genocide.
> The damage to the environment Perplexity and Google (and Kagi) cause with unnecessary AI usage is a much bigger concern to me personally.
Historically interesting note: Nazi Germany kidnapped thousands and thousands of Polish children and handed them to German parents to be raised as Germans. Their decedents usually don't even know about this.
Back to the topic at hand: you have to distinguish between material and formal cooperation with evil. I don't know the Kagi situation (if they're just indexing images for Yandex to improve search results, then I don't see how you have a real case here; even calling this remote cooperation with evil -- something that is generally impossible to avoid -- seems like a stretch). But let's say a company is doing something like making financial contributions to some organization doing something immoral. While you can boycott a company for that reason, you are not generally morally obligated to do so. And in practice, it usually has no effect. It's also unjust to saddle people with a burden of guilt they do not actually have. This is called rigorism.
Sorry, but this is not really enough of a concern to care about 2% of a $5 monthly fee going towards a company (Yandex) whose involvement with Russia's war seems iffy (Russia themselves fined the company for refusing to give user data to its state intelligence).
>Multiple things can be damaging at once.
Yes, but the point is that if I can tolerate some of the money Google or Amazon gets from me going to fund concentration camps abroad, in which camp residents receive a quarter of the calories per day (~250) that victims of Nazi concentration camps did, then I can tolerate this. Everyone has to draw a line somewhere, and I see no reason to draw it at Kagi but not Google/AWS/Microsoft/Apple/etc.
We have an organic channel like this that's just called "Study Hall". People constantly ask technical questions and they know it's a judgement free zone. Probably one of the most productive chat channels in our org.
that's the thing that's so inorganic about this whole thing : it's not a judgement free zone, it's a zone that tricks people into presuming that.
If some underling somewhere says something that exposes their ignorance or naivety to either a policy problem or a technical problem you'd better realize that it's going to trigger a 'review mechanism' somewhere down the road within the organization; to think otherwise would be pure fantasy.
Similarly : if you go drinking with the boss, you do still have to remember that the drunk puking slob who you're carrying to their hotel room is going to wake up and be your boss tomorrow.
very few humans actually disconnect this stuff from their internalized judgements of people.
> that's the thing that's so inorganic about this whole thing : it's not a judgement free zone, it's a zone that tricks people into presuming that.
I am surprised that I had to scroll this far to find this observation. If I see a person's posts in multiple channels, I don't switch ON and OFF my mental image of them based on channel like a SQL where clause where channel <> personal. Based on the same premise, the person posting also probably knows this fact and isn't going to be totally free and relaxed asking whatever comes to their mind due to fear of judgement - the same eyeballs are going to be looking at the posts, irrespective of the type of channel.
As someone posted elsewhere, it is all going to be cultural and if that is right, it pretty much doesn't matter how you structure channels.
Yeah, maybe I’m small-minded, but if someone I’m not familiar with, say a new hire asks a question way beneath their presumed experience level I’m absolutely gonna judge, judgement free be damned; and if they’re my report I’m gonna question the hiring (in my mind). There’s no shortage of imposters in the industry, most of them who’re capable of landing jobs above them are probably also smart enough to scoff at pure fantasy like “judgement free zone”.
Having spent a long time in tech and worked with a lot of people I've realised there are two sorts of people who are "imposters". There are those who have BS'd their way up and are in a role where they're out of their depth, and there are those who were lucky to have landed a role that's a bit beyond them (often because they have deep experience elsewhere.)
The first type don't ask questions. They know they're imposters and don't want to be called out.
The second type do ask questions. They also know they're imposters but they're trying to learn so they're not.
Judge people on their actions else you'll only spot the second group, and often with a bit of support those ones can go on to do great things, especially if they're experienced at one thing but they're not learning a new thing. When they get enough knowledge to connect the two things they can be absolutely brilliant.
There's no such thing as a judgment free zone when humans are involved :-)
I tell new hires that they shouldn't be scared of asking questions, and that if they're not asking questions they're probably not pushing themselves enough. But also caution to make sure that they check available resources first, and then ask the right audience.
I'm a tech lead and I also tried to encourage the judgement free zone on the devs channel. I do this by intentionally asking question that I should know the answer to but don't as I just never got around to figuring it out as I have 16k other things to worry about. So rather than spend 2 hours chasing something down, I just ask. I know that this is a question a lot of juniors can answer. I avoid people thinking I am dumb by also taking time to answer the more technical questions and posting architecture and optimizations on the same channel.
This strikes me as a somewhat unfair characterization of many of these communities. In my experience, a much more common issue is that the people who do have answers end up ignoring the group and it becomes pointless. It rarely becomes a source of career hindrance or long-lasting judgement, it just ends up being useless because there's not a lot of incentive for the expert side of the equation.
People who are likely to judge people for dumb questions are rarely involved in those groups in the first place, for exactly all the obvious reasons.
The more realistic outcome isn’t that your boss ends up a drunk puking slob (and for what it’s worth most of these groups don’t include leadership anyway, so not sure why anyone's boss would be involved) but that an intern floats a terrible idea ("I'm thinking of taking these 10 shots of 151"), nobody responds, they take silence as approval, and they end up causing a mess and then being judged for the mess they caused.
A quick gut check from them with a healthy group might get a few eye rolls and a "here's why that's a bad idea", but not any lasting judgement unless they completely ignored the advice.
The only case I can think of where that might happen is if they already did something which has policy or legal implications ("hey i accidentally dumped the whole user base including PII to my phone"), in which case - good? There should be a review mechanism, including consequences if they ignored a bunch of roadblocks.
> It rarely becomes a source of career hindrance or long-lasting judgement, it just ends up being useless because there's not a lot of incentive for the expert side of the equation.
Yeah, the incentive structure for something like this is totally misaligned for this to work effectively in many cases outside of a very small, tight-knit team. (In which case... why the formality in the first place?)
For the "juniors": Why waste time digging through documentation, searching, or thinking--I can just post and get an answer with less effort.
For the "seniors": I'm already busy. Why waste time answering these same questions over and over when there's no personal benefit to doing so?
Sure, there are some juniors that will try and use it as a last resort and some seniors that will try their best to be helpful because they're just helpful people... but I usually see the juniors drowned out by those described above and the experts turn into those described above.
I think we _could_ come up with something that better aligned incentives though. Spitballing--
Juniors can ask a question. Once a senior answers, the junior then takes responsibility for making sure that question doesn't need to be answered there again--improving the documentation based on that answer. Whether that's creating new documentation, adding links or improving keywords to help with search, etc. That change then gets posted for a quick edit/approval by the senior mainly to ensure accuracy.
Now we're looking at something more like:
For the "juniors": If I ask a question, I will get an answer but it will create additional work on my end. If I ask something already answered in the documentation that I could have easily found, I basically have to publicly out myself as not having looked when I can't propose an improvement to the documentation. And that, fairly, is going to involve some judgement.
For the "seniors": Once I answer a question, someone is going to take responsibility for getting this from my head into documentation so I never need to answer this again.
This has an added benefit of shifting some of the documentation time off of the higher paid, generally more productive employees onto the lower paid, less productive employees and requiring them to build out some understanding in order to put it into words. It may also help produce some better documentation because stuff that a senior writes is more often going to assume knowledge that stuff a junior writing may think to explain because _they_ didn't know it. It also means that searching in the Slack/other channel, any question you find should end up with a link to the documentation where it's been answered which should help you discover more adjacent documentation all of which should be the most up-to-date and canonical answer we have.
I’m on board with the overall point, though I’d actually flip the logic in this section:
> Once a senior answers, the junior then takes responsibility for making sure that question doesn't need to be answered there again.
That might make sense for simple questions. But for anything more complex, especially when the issue stems from something you have control over, having senior folks take ownership might make more sense. If they can tie the fix to visible impact, there’s a strong incentive for them to actually solve the root problem. Otherwise, there’s always the risk that experienced team members simply ignore the question 100% of the time (which also solves the problem of "i've already answered this question").
One way seniors might approach these types of groups is by treating them as a source of ideas. Repeated questions like “how do I use X?” might indicate that X needs a redesign or better onboarding. An experienced corporate climber could treat those questions as justification for "X 2.0 which is way easier to onboard to" and get backing to work on it.
Anyone who’s spent time at a large tech company has likely seen this dynamic play out, because it’s a common pathway to promotion. Definitely taken to problematic extremes, no doubt, but a slightly-healthier version of that playbook still beats the alternative of relying on the arcane knowledge of a select few as gatekeepers of information.
As a senior judgement is a real problem as others have suggested. I suspect its a reason why AI is popular. I can ask those dumb questions I should know without embarrassing myself by asking someone, or even worse put it in writing with my name on it.
You'd have to do some styling, but Plotly is what I'd use to recreate the graphs on the gallery page there. It would obviously require some work to match the exact styling, but it should all be doable and TTFP would probably be much lower assuming you're starting with a Plotly express template.
Kind of unfortunate that it uses pyright and jedi instead of just basedpyright for the more advanced features. Python language support just isn't great with jedi compared to pylance or basedpyright.
And not to beat a dead horse, but I'm also not a huge fan of the broad claims around it being OSS when it very clearly has some strict limitations.
I've already had to migrate from R Connect Server / Posit Server at work, because of the extreme pricing for doing simple things like having auth enabled on internal apps.
We found a great alternative that's much better anyways, plus made our security folks a lot happier, but it was still a massive pain and frustrated users. I've avoided any commercial products from Posit since then and this one makes me hesitant especially with these blurry lines.
They definitely target large scale companies, but you can use their SaaS offering and it can be relatively affordable. The best part is the flexibility and scaling, but the license model is awesome too. There's no usage based billing, you just pay a flat license fee per user that writes code and for the underlying cloud costs and they'll deploy it on GCP, AWS, or Azure.
They're used by a lot of large companies, but academia as well to replace or augment on-prem HPC clusters. That's what we used them for as well.
It's a shame that they don't have you writing marketing copy! The docs are indeed a lot more reasonable looking (to me at least). I work for a small proprietary fund and not some Godzilla company these days so maybe I'm just not the audience, but whew, for purchasing decision makers with subject matter background, that home page would have been a back button real fast if it wasn't linked from your thoughtful comment.
I'm interested in your opinion as a user on a bit of a new conundrum for me: for as many jobs / contracts as I can remember, the data science was central enough that we were building it ourselves from like, the object store up.
But in my current role, I'm managing a whole different kind of infrastructure that pulls in very different directions and the people who need to interact with data range from full-time quants to people with very little programming experience and so I'm kinda peeking around for an all-in-one solution. Log the rows here, connect the notebook here, right this way to your comprehensive dashboards and graphs with great defaults.
Is this what I should be looking at? The code that needs to run on the data is your standard statistical and numerics Python type stuff (and if R was available it would probably get used but I don't need it): I need a dataframe of all the foo from date to date and I want to run a regression and maybe set up a little Monte Carlo thing. Hey that one is really useful, let's make it compute that every night and put it on the wall.
I think we'd pay a lot for an answer here and I really don't want to like, break out pyarrow and start setting up tables.
I'll just say Domino presents very much as a code first solution. So, if you want staff to be able to make dashboards _without_ code like using Looker Studio, then this isn't it.
The one other big thing that Domino isn't, is it's not a database or data warehouse. You pair it with something like BigQuery or Snowflake or just S3 and it takes a huge amount of the headache of using those things away for the staff you're describing. The best way to understand it is to just look at this page: https://docs.dominodatalab.com/en/cloud/user_guide/fa5f3a/us...
People at my work, myself included, absolutely love this feature. We have an incredibly strict and complex cloud environment and this makes it, so people can skip the setup nonsense and it will just work.
This isn't to say that you can't store data in Domino, it's just not a SQL engine. Another loved feature is their datasets. It's just EFS masquerading as an NFS, but Domino handles permissions and mounting. It's great for non-SQL file storage. https://docs.dominodatalab.com/en/cloud/user_guide/6942ab/us...
So, with those constraints in mind, I'd say it's great for what you're describing. You can deploy apps or API endpoints. You can create on-demand large scale clusters. We have people using Spark, Ray, Dask, and MPI. You can schedule jobs and you can interact with the whole platform programmatically.
Looks like we're in a similar situation. What is your current go-to for setting up lean incremental data pipelines?
For me the core of the solution - parquet in object store at rest and arrow for IPC - haven't changed in years, but I'm tired of re-building the whole metadata layer and job dependency graphs at every new place. Of course the building blocks get smarter with time (SlateDB, DuckDB, etc.) but it's all so tiresome.
Yeah, last time I had to do this was about a year ago and I used parquet and arrow on S3-compatible object stores and put a bunch of metadata in postgres and the whole thing. At that time we used Prefect for orchestration which was fine but IMHO not worth what it cost, I've also used flyte seriously and dabbled with other things, nothing that I can get really excited about recommending, it's all sort of fine but kinda meh. I used to work for a megacorp with extremely serious tooling around this and everything I've tried in open source makes me miss that.
On the front end I've always had reasonable outcomes with `wandb` for tracking runs once you kind get it all set up nicely, but it's a long tail of configuration and writing a bunch of glue code.
In this situation I'm dealing with a pretty medium amount of data and very modest model training needs (closer to `sklearn` than some mega-CUDA thing) and it feels like I should be able to give someone the company card and just get one of those things with 7 programming languages at the top of the monospace text box for "here's how to log a row", we do Smart Things and now you have this awesome web dashboard and you can give your quants this `curl foo | sh` snippet and their VSCode Jupyter will be awesome.
Just reading this as well and I neglected to mention that the Domino thing we use has Flyte (They call it Flows, but it's the same thing) and MLFlow built-in as well.
We do discount heavily for academia: get 50% off for research and 100% off (i.e. free) for teaching. But I do get that our pro products largely solve problems that folks encounter in larger enterprises, and you may not see the value inside an academic department. I'm also always happy to learn how we could do better, please feel free to reach out to [email protected].
Thank you for the response! My key recommendation is to unbundle. At one point we have been told "It got 35% more expensive, but it does so much more now it supports Python" - we didn't ask for Python. To effectively use Connect with private packages you need this other full-blown institutional Package Manager license, 99% of your users will not need. Also per-named-user pricing (rather than per active seat) is so aggressive. A user uses a shiny app once in a year, still need a full per year license.... I feel Posit is one of the most agressive companies in terms of upselling, while positioning as this benevolent PBC / Open-Source institution - just go full Oracle at least we will know where things stand.
Positron looks like the next version of Rstudio, which is currently free. Do you think the plan is to phase out support for the free product and push users into the paid one?
Positron inherits many ideas from RStudio, but is a separate project with an intentionally different set of tradeoffs; it gains multi-language/multi-session support, better configuration/extensibility, etc. but at the expense of RStudio's simplicity and support for many R-only workflows.
We're still investing in RStudio and while the products have some overlap there's no attempt to convert people from one to the other.
I am talking about the RStudio Server and Connect - these are really expensive. One of the sales reps claimed that it is so expensive because they are a PBC and support open-source development. As in if they were just for profit it would be cheaper, but we should feel good about paying more. I could not take it.
As an admin and advocate for Posit Teams, Connect and Server filled a niche where a single admin could spin up infra and allow for anything deployable by end users without having to worry about scaling.
It paid for itself in terms of scientists spinning up their own projects without having to provision server hardware, VMs, or anything else.
If you take advantage of all the featuers in Teams, perhaps. But we needed a tiny bit from Workbench, a little bit from Connect, and sliver from Package Manager - and Teans ended up eating a huge portion of our IT bill, effectively stunting our efficiency as a research organization. Over the years, while our use case did not change, our Posit bill more than doubled.
My friend and I are so excited about this bot that we're actively looking for AI grants to apply to for funding the purchase! The price is incredible for what you get, but we both work in the public sector :/
Consider the ROS simulators first, and look at the platforms used in RoboCup events. There are better platforms than Talos or HECTOR around.
It can take years to get a basic bipedal platform operational, and in general it takes 3 times as long to tune the software/firmware. Unless you see actual proof of platform operation for more than a minute, than take any claims as marketing hype.
Could also try a cheap servo hexapod or turtle-bot kit first, as stable well-studied platforms are easier to code on. =3
