> To me, it felt like the feature was presented as a done deal because of security.
Not security, but integrity, although security (which is the #1 concern of companies relying on a platform responsible for trillions of dollars) is certainly one of the primary motivations for integrity (others being performance, backward compatibility or "evolvability", and correctness). Integrity is the ability of code to locally declare its reliance on some invariant - e.g. that a certain class must not be extended, that a method can only be called by other methods in the same class, or that a field cannot be reassigned after being assigned in the constructor - and have the platform guarantee that the invariant is preserved globally throughout the lifetime of the program, no matter what other code does. What we call "memory safety" is an example of some invariants that have integrity.
This is obviously important for security as it significantly reduces the blast radius of a vulnerability (some attacks that can be done in JS or Python cannot be done in Java), but it's also important for performance, as the compiler needs to know that certain optimisations preserve meaning. E.g. strings cannot be constant-folded if they can't be relied upon to be truly immutable. It's also important for backward-compatibility or "evolvability", as libraries cannot depend on internals that are not explicitly exposed as public APIs; libraries doing that was the cause of the migration pain from Java 8 to 9+, as lots of libraries depended on internal, non-API methods that have changed when the JDK's evolution started picking up steam.
In Java, we've adopted a policy we call Integrity by Default (https://openjdk.org/jeps/8305968), which means that code in one component can violate invariants established by code in another component only if the application is made aware of it and allows it. What isn't allowed is for a library - which could be some fourth-level dependency - to decide for itself at some point during the program's execution, without the application's knowledge, that actually strings in this program should be mutable. We were, and are, open to any ideas as long as this principle is preserved.
Authors of components that do want to do such things find the policy inconvenient because their consumers need to do something extra that isn't required when using normal libraries. But this is a classic case of different users having conflicting requirements. No matter what you do, someone will be inconvenienced. We, the maintainers of the JDK, have opted for a solution that we believe minimises the pain and risk overall, when integrated over all users: Integrity is on by default, and components that wish to break it need an explicit configuration option to allow that.
> built on a solid foundation with ByteBuddy
ByteBuddy's author acknowledges that at least some aspects of ByteBuddy - and in particular the self-loading agent that Mockito used - weren't really a solid foundation, but now it should be: https://youtu.be/AzfhxgkBL9s?t=1843. We are grateful to Rafael for explaining his needs to us so that we could find a way to satisfy them without violating Integrity by Default.
Is there a point at which library maintainer feedback would meaningfully influence a by-default JVM change?
I keep a large production Java codebase and its deployments up-to-date. Short of upstreaming fixes to every major dependency, the only feasible way to continue upgrading JDK/JVM versions has often been to carry explicit exceptions to new defaults.
JPMS is a good example: --add-opens still remains valuable today for important infra like Hadoop, Spark, and Netty. If other, even more core projects (e.g. Arrow) hadn't modernized, the exceptions would be even more prolific.
If libraries so heavily depended upon like Mockito are unable to offer a viable alternative in response to JEP 451, my reaction would be to re-enable dynamic agent attachment rather than re-architect many years of test suites. I can't speak for others, but if this reaction holds broadly it would seem to defeat the point of by-default changes.
> Is there a point at which library maintainer feedback would meaningfully influence a by-default JVM change?
Of course, but keep in mind that all these changes were and are being done in response to feedback from other users, and we need to balance the requirements of mocking frameworks with those of people asking for better performance, better security, and better backward compatibility. When you have such a large ecosystem, users can have contradictory demands and sometimes it's impossible to satisfy everyone simultaneously. In those cases, we try to choose whatever we think will do the most good and the least harm over the entire ecosystem.
> JPMS is a good example: --add-opens still remains valuable today for important infra like Hadoop, Spark, and Netty. If other, even more core projects (e.g. Arrow) hadn't modernized, the exceptions would be even more prolific.
I think you have answered your own question. Make sure the libraries you rely on are well maintained, and if not - support them financially (actually, support them also if they are). BTW, I think that Netty is already in the process of abandoning its hacking of internals.
Anyone who has hacked internals agreed to a deal made in the notice we had in the internal files for many years prior to encapsulation [1], which was that the use of internals carries a commitment to added maintenance. Once they use the new supported mechanisms, that added burden is gone but they need to get there. I appreciate the fact that some open source projects are done by volunteers, and I think their users should compensate them, but they did enter into this deal voluntarily.
> If libraries so heavily depended upon like Mockito are unable to offer a viable alternative in response to JEP 451
The main "ergonomic" issue was lack of help from build tools like Gradle/Maven.
[1]: The notice was some version of:
WARNING: The contents of this source file are not part of any supported API.
Code that depends on them does so at its own risk: they are subject to change or removal without notice.
Of course, we did end up giving notice, usually a few years in advance, but no amount of time is sufficient for everyone. Note that JEP 451 is still in the warning period that started over two years ago (although probably not for long)
The auto-attach flag isn’t a huge deal, since it’s a one-liner that can be statically documented and the fix works in all cases. The bigger issue is the JDK / runtime team’s stance that libraries should not be able to dynamically attach agents, and that the auto-attach flag might be removed in the future. You can still enable mockito’s agent with the —javaagent flag, but you have to provide the path to the mockito jar, and getting that path right is highly build-system-dependent and not something the mockito team can document in a way that minimizes the entry threshold for newcomers.
I haven't read the discussion but this seems like the obvious answer considering the flag only needs to be set during test.
Presumably this might miss some edge case (where something else also needs the flag?) though an explicit allow of the mockito agent in the jvm arg would have solved for that.
In September there was a supply-chain attack on NPM where the payload code injected hooks into the DOM API. Changing the behaviour of encapsulated components, like Java's standard library, is not possible now without the application explicitly allowing code to break the integrity of the encapsulated component.
Not security, but integrity, although security (which is the #1 concern of companies relying on a platform responsible for trillions of dollars) is certainly one of the primary motivations for integrity (others being performance, backward compatibility or "evolvability", and correctness). Integrity is the ability of code to locally declare its reliance on some invariant - e.g. that a certain class must not be extended, that a method can only be called by other methods in the same class, or that a field cannot be reassigned after being assigned in the constructor - and have the platform guarantee that the invariant is preserved globally throughout the lifetime of the program, no matter what other code does. What we call "memory safety" is an example of some invariants that have integrity.
This is obviously important for security as it significantly reduces the blast radius of a vulnerability (some attacks that can be done in JS or Python cannot be done in Java), but it's also important for performance, as the compiler needs to know that certain optimisations preserve meaning. E.g. strings cannot be constant-folded if they can't be relied upon to be truly immutable. It's also important for backward-compatibility or "evolvability", as libraries cannot depend on internals that are not explicitly exposed as public APIs; libraries doing that was the cause of the migration pain from Java 8 to 9+, as lots of libraries depended on internal, non-API methods that have changed when the JDK's evolution started picking up steam.
In Java, we've adopted a policy we call Integrity by Default (https://openjdk.org/jeps/8305968), which means that code in one component can violate invariants established by code in another component only if the application is made aware of it and allows it. What isn't allowed is for a library - which could be some fourth-level dependency - to decide for itself at some point during the program's execution, without the application's knowledge, that actually strings in this program should be mutable. We were, and are, open to any ideas as long as this principle is preserved.
Authors of components that do want to do such things find the policy inconvenient because their consumers need to do something extra that isn't required when using normal libraries. But this is a classic case of different users having conflicting requirements. No matter what you do, someone will be inconvenienced. We, the maintainers of the JDK, have opted for a solution that we believe minimises the pain and risk overall, when integrated over all users: Integrity is on by default, and components that wish to break it need an explicit configuration option to allow that.
> built on a solid foundation with ByteBuddy
ByteBuddy's author acknowledges that at least some aspects of ByteBuddy - and in particular the self-loading agent that Mockito used - weren't really a solid foundation, but now it should be: https://youtu.be/AzfhxgkBL9s?t=1843. We are grateful to Rafael for explaining his needs to us so that we could find a way to satisfy them without violating Integrity by Default.