You likely don't need to optimize anything; Emacs has seen some pretty significant optimizations recently (native Emacs Lisp compilation, tree-sitter modes, better handling of long lines, etc.) so performance is rarely the issue.
However, you do need to avoid call-process (spawning blocking processes) as much as possible. Also, my experience with TRAMP has been pretty awful due to the fix for https://debbugs.gnu.org/cgi/bugreport.cgi?bug=12145 (literally: TRAMP blocks all of Emacs while waiting on a network connection).
If you want to try eshell, try combining it with EAT (eat-eshell-mode).
> Maybe in my retirement I will end my career by helping to make emacs + EXWM multi threaded. I am guessing that is a daunting project, but it sure would be fulfilling.
This isn't fixable with threads, unfortunately. The issue is that:
1. Emacs e.g., launches a process with call-process. This blocks EVERYTHING (including other threads).
2. That process wants to map the window but EXWM can't respond to this request because Emacs is blocked.
3. The call to call-process never returns because the process can't create its window.
At this point, I think the right answer is to write a minimal out-of-process window manager (e.g., a wayland compositor).
1. During normal operation, it would behave like EXWM and ask Emacs how to manage windows, etc.
2. In special cases (TBD), it would behave autonomously, acting like a standard floating window manager until Emacs becomes responsive again.
IMO, they're a great way to get started without having to invest too much time up-front. On the other hand, that was 10 years ago and it's a LOT easier to throw together a usable config nowadays; with LSP + built-in tree-sitter modes, you no longer need 3 packages per language plus a bunch of configuration glue.
I used to do this. What finally killed it wasn't reputation, it was the fact that I needed 100% uptime or risk losing messages, getting my address blacklisted, etc. Email is supposed to be resilient to down time (retries, trying each MX record, etc.) but I found that large mail providers tend to just bounce and walk away.
Worse, GitHub (back in 2016 and 2018) would mark a recipient as "unavailable" after a single bounce, refusing to send any more notifications to that address. They since improved the situation and their support was actually very helpful and responsive here, but it's pretty clear that modern SMTP senders have an expectation that recipients will be "always online" that didn't exist when the protocol was invented.
I have a feature (called greylisting) whereby my server intentionally rejects the first mail it receives from a domain.
I have never had anyone claim that their mail has not been delivered to me, and I get a lot of mail.
Retry is built in to the spec, and if you’re really worried you can put a second “receive” SMTP server on the internet with a lower priority, and have it backhaul with LMTP.
———
Email was designed in a time where hosts were not perpetually connected to each other.
Gmail always rejects the first email I send to a new gmail account. It does this every time – and has done for years – despite the fact I have sent emails to hundreds of other gmail accounts, and send emails to such accounts every day.
This is the reason I personally will not touch any Google services. And in business, I excise Google services as a priority. If a company cannot handle email in a civil manner, it certainly can't be trusted with anything of importance.
> Q: If your server(s) is/are offline for a few hours, why would you "lose messages"?
They said...
>> Email is supposed to be resilient to down time (retries, trying each MX record, etc.) but I found that large mail providers tend to just bounce and walk away.
I take that to mean that if your server isn't availble to receive the mail at the time it is first offered, it won't be retried later. That wasn't the case (for most mail) when I gave up on self hosting 10 years ago, but it's plausible.
It's not reasonable. Mail not deliverable is not the same as house burned down, recipient moved unknown or sth, it simply means the letter was not received. Who and why messed up is unknown, thus NO mail server will mark you down after a single attempt.
This is fearmongering. My mails always got resent after some hours or a day. It's absolutely NOT possible to tell if the problem is on your side, senders side or somewhere in between why a mail is not delivered once and no standard server config would simply toss it.
Host your own mail. I get 99% deliverability with 0 repuation since i do dkim and spf correct.
Don't be distracted by the "complexity" - if you config right it's totally doable.
>I get 99% deliverability with 0 repuation since i do dkim and spf correct.
Your anecdote of success doesn't matter to the others that correctly configured DKIM/SPF and still don't get their emails delivered to Gmail/Outlook/Yahoo/etc. E.g. : https://news.ycombinator.com/item?id=32715437
One of the reasons for hard-to-diagnose sending failures is that Gmail/Outlook have "extra invisible rules" that override correct DKIM/SPF settings because spammers and phishers also have correct DKIM/SPF. So they use extra heuristics such as "ip reputation" etc.
And even after one gets it working, e.g. "submit some form" to Microsoft and wait a few days to get things unblocked... the deliverability may break again because of another "invisible heuristic".
EDIT to reply: >No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Delivery reliability can still break without using a relay.
In fact, this unreliability of 100% self-hosting at home is why some self-hosters split it into a hybrid setup and add an external relay for outgoing SMTP and only keep self-hosting for receiving email.
Get this. I owened a /23 for 7 years (still own it today) and kept the mail server ip on a /27 just for the mail server on a /24 that was not used for anything production (firewalled and maybe 3 ip's responded on port 443). My mails were banned for bad reputation. The provider which hosted my /23 was well known for responding to abuse, even falsely flagging my account as abusive in the early days for simply _sending_ valid smtp mails.
IP reputation turned out to mean, if they never saw your IP, you were in the banned bucket. How do you even fight against that
No, that's because your relay overwrites part of the header which makes dkim strict break. Change to relaxed or don't modify the header on your relay.
Outlook business will accept your mail, Outlook private may filter, but the rates fluctuate so heavy i suspect its rules based on user behaiviour/interests. I dono, cant have both spamfree inbox and 0 false positives.
I think i found a loophole for the google and outlook ones... I have had my domains on both providers, and then left to my own (but left a couple of google and ms txt records by mistake) and never had any issues delivering to both providers. Thinking of doing the same thing again honestly, but looking at good providers at the moment.
I hate the fact that your comment got flagged / greyed out / whatever even though it's perfectly correct. I'm one of those people who had configured everything perfectly. Score of 100 on mail-tester, SPF, DKIM, DMARC, you name it. Examining the headers in an e-mail sent to gmail: pass, pass, pass. Everything green.
Microsoft however? Denied, 100% of the time. Spam folder, or even plain rejected. Why? No idea, they won't say. They redirect you to their shitty partner that you can PAY in order to HOPE you get approved.
I don't know why our experiences are considered "anecdotes", and not the other way round. What's the incentive for big players to accept e-mail from home servers or small dedicated servers? "Sure it could be Standard Nerd from HN running their own stuff for street cred points, or it could be one of the bazillion spam factories sending fake UPS scams. In doubt, let's reject."
I add it here so you can successful self-host: You need strict DMARC for Microsoft. If you change the header on your relay DMARC relaxed filters will pass the mail, but not strict.
Because this adds the need to sign every single mail for every single recipient (expensive) its safe to filter for this as a SPAM-Server will sign mail once, then distribute.
That's why your mail is filtered - not because your non-blacklisted IP is the problem or whatever.
>I hate the fact that your comment got flagged / greyed out / whatever even though it's perfectly correct. [...] I don't know why our experiences are considered "anecdotes", and not the other way round.
It's because people who successfully self-host think their situation universally applies to everyone.
Here's another example from 2017 of someone replying to my previous reasonable comment about self-hosting by overconfidently saying I was exaggerating the issues : https://news.ycombinator.com/item?id=15526127
So they end up solving it by "outsourcing" the outbound email to a relay (SendGrid).
So my comment gets downvoted for explaining what others had to do in the real world.
The following should not be a controversial statement but for some reason it is: Correctly configuring SPF/DKIM/DMARC and getting 100% green score on https://www.mail-tester.com/ for your self-hosted setup ... does not universally mean your outbound email will get accepted by all the services.
Read the logs from Gmail and Microsoft, they will tell you exactly why the mail was filtered. Act on that problem and have your mail appear in inboxes.
It's usually relaxed DMARC triggering Microsoft. Gmail accepts relaxed.
The recommendation to resolve from handles to DIDs for "permalinks" is concerning to me:
- My handle is something _I_ control. I can make it point at a different PDS at any time.
- My DID is something my PDS controls.
I could solve this by indirecting through a web DID under my control, but there's no recommendation anywhere in Bluesky's documentation. Is that something everyone needs to do to ensure real identity portability?
edit: I'm not sure this CAN be solved without running a PDS given that I can't use my own keys. What am I missing here?
What you control is your identity (i.e. DID Document). As long as you control your identity, you can change either your handle or your hosting aka PDS.
What stebalien might be referring to is that your DID document for PLC specifically is controlled by anyone with the rotation key.
In the Bluesky implementation, this is Bluesky for convenience’s sake, to make it possible for users to easily sign up. (I’m not sure internally if it’s part of the PDS or held separately.)
PLC has a mechanism allowing “higher” keys to override “lower” ones within a certain time window, so being able to add your own rotation key that “outranks” Bluesky’s would solve this issue.
Alternatively, use web DIDs and then it’s fully self-managed just as DNS would be.
This is mostly separate to the PDS - Bluesky is more like the client here (albeit, they also currently run the PLC service). It’s part of your DID document which they manage for you mostly, but there’s ways to take ownership of your PLC DID - see Dan’s links for that.
For our non-atproto uses of the PLC directory, we have similar needs, and we’ll likely let users provide their own public key before we create their document to help solve this. We have a technical audience though, so that solution may not make sense for Bluesky - but there’s a lot of people thinking about how to improve this in the atmosphere.
From my reading of your blog post, it sounds like the DID is the ultimate authority and not my domain name, which sounds like a pretty big problem for user portability.
Ah, that's exactly what I was looking for. Thanks!
I guess I get why it works that way (avoids some issues with domain expiration) but... honestly, I'd rather have my domain name in control. Even after registering my own rotation key, I'm still at the mercy of the centralized PLC directory.
Unfortunately, it looks like it's not possible to migrate to a web DID without starting over.
So the reason to have the DID in control is for users that say sign up for Bluesky and have a bluesky handle which is a domain they don't control and want to move to a different provider without breaking everything. If your handle was the thing in control then you must own a domain to own your account and move providers. That kinds of defeats the purpose of migratable PDSs for the "rest of us" use case.
Yeah, you can't migrate a DID. You could bulk-import the content (and even fix internal links within it) but that wouldn't change the links pointing at the old DID.
Or one can just use semantic HTML; it's easy enough to convert semantic HTML into markdown with a tool like pandoc. That would also help screen readers, browser "reader modes", text-based web browsers, etc.
I still haven't seen anyone discuss the issues with distributing applications containing GPLv3 components under these new rules given the clause (from the GPLv3):
> “Installation Information” for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.
At the moment, the workaround here is that keys can technically just be generated on the fly (with some caveats). With Google's new requirements, that's not possible.
In my interpretation, this clause is for when someone ships a user product that contains GPLv3 software. That means it would apply to the phone vendor if the phone contained GPLv3 (or anything using LGPLv3) software.
But if you're just a developer who ship software GPLv3 software for Android, you are good because any developer that want to modify your software on their phone can, as long as they register to Google to get these keys. It should therefore be respecting the licenses.
Sure, but that means that either Google or the application author would be required to give me working keys with no restrictions, which would make the entire system rather pointless.
However, now that I think about it, the fact that "unauthorized" apps can still be installed via ADB exception may cover this?
> as long as they register to Google to get these keys
As soon as e.g. an Iranian user gets access to your GPLv3 app, you've got a problem. They cannot register with Google (due to sanctions), but you are responsible for ensuring they can install and distribute their modified app just as you have.
They aren't responsible for ensuring that others can install it.
That part of GPLv3, commonly called the "anti-Tivoization" clause, only applies if you "convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized)".
This was narrowly written to only cover situations like Tivo, which was a hardware vendor locking down GPL code on the hardware they sold.
I couldn't find such requirements when reading the GPL.
The paragraph cited by GP is from the explicitly about "convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term". So in other words, only if you sell hardware with binaries under GPL.
Also, from reading other comments, it seems it would still be possible to use the adb console to load apps without having signatures? So that should cover it as far as the GPL is concerned.
IANAL but isn't that the purpose of the passage below (emphasis mine)? I agree it's subject to interpretation whether the license also allows one to provide detailed instructions on how to obtain new keys from a third party and install the application using them. However, it seems to me the passage implies that if Google is to deny someone developer keys and installation of the modified application, then the original distributor of the application is in violation of the GPLv3.
----
'“Installation Information” for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.
If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information.'
But the "original distributor of the application" is not convoying the binary in "a transaction in which the right of possession and use of the User Product is transferred to the recipient", so that clause doesn't apply.
In this context, the "User Product" would be the phone, as defined in the previous paragraph of the license.
I do think that this very much puts Google in the same boat as Apple in terms of how the GPL is deemed compatible or not for distribution to their platforms and proprietary stores.
Personally, I think that the GPL is still compatible with both platforms, as I've written about before[1]. There's plenty of GPL software on both the Play Store and App Store (Signal, Element, Wordpress, SimpleNote, Bitwarden, Mastodon, Telegram, and Proton Mail, just to name a few), but people tend to feel that iOS is a more hostile environment. The mandatory developer registration requirement may bring a more even-handed assessment of how the GPL and these app stores can live together.
1. Release binary-only updates (opt-in).
2. Let the community (a) make GPL source requests for any GPLed components and (b) let the community reverse engineer the vulnerabilities from the binary updates.
3. Publish the source once everything is public anyways.
Which just shows how utterly ridiculous all this is.
However, you do need to avoid call-process (spawning blocking processes) as much as possible. Also, my experience with TRAMP has been pretty awful due to the fix for https://debbugs.gnu.org/cgi/bugreport.cgi?bug=12145 (literally: TRAMP blocks all of Emacs while waiting on a network connection).
reply