Tuesday August 16, 2011

Blog moved!!!

This blog has moved!

I have moved the history of posts and comments from JRoller to the new location of blog.joda.org. However, I can't find a way to auto-redirect the feed.

Please resubscribe to the new blog using this feed:
http://blog.joda.org/feeds/posts/default.

Sorry for the resubscribe inconvenience!

Monday August 01, 2011

Joda-Time v2.0

Joda-Time version 2.0 is now released! Download. Release notes.

Firstly, Joda-Time 2.0 is NOT a re-write of Joda-Time. The major version number change signifies that there are some incompatibilities. However, these are as minor as I could make them.

A few methods on Chronology that were deprecated for years have been removed, but most other deprecated and classes methods remain as the benefits of removing them were lower than the damage caused.

The library has moved to JDK 1.5 compatibility, adding generics. Sadly, adding generics to Comparable in some cases caused issues for implementations. The choice was to aim for binary incompatibility at the expense of source incompatibility if the ReadableInstant or ReadableDuration interfaces were implemented directly. If you didn't implement these, or extended the supplied abstract class then you should have no problems. Similarly, ReadablePartial had Comparable added, but if you didn't implement it, or extended the supplied abstract class then again you should have no problems.

In summary, I hope there are no significant binary incompatibilities. If you do find a binary incompatibility that I didn't, I'd like to know, particularly if it affects another open source project. If it is in the next week or so, then perhaps the incompatibility can be fixed and "jar hell" avoided.

There are a number of semantic changes as well in the corner cases. Again see the release notes.

• Parsing a month-day without a year now selects year 2000 rather than 1970 (so the 29th of February can be parsed)
• Edge cases with a week-based-year and a month are handled better
• Methods that handle daylight savings time (DST) are more consistent, retaining the offset wherever possible during calculations or consistently choosing summer time if not
• Some short time zone IDs have been changed
• DateTimeZone factory taking an ID is now case sensitive

And of course there are some enhancements:

• Static factories now() and parse() added to the main date-time classes
• New classes YearMonth and MonthDay
• API added to plugin to the Joda-Time clock
• Period formatting in multiple langauges
• Parsing of LocalDate/LocalTime/LocalDateTime directly supported (much better than previously)
• Parsing of time-zone IDs and some support for parsing time-zone names (via the builder)
• Better handling of 'Z' when parsing to mean +00:00
• Reliable way to get the start of day for a DateTime - withStartOfDay()
• Convenient way to handle DST, DateTime.withEarlierOffsetAtOverlap() .withLaterOffsetAtOverlap()
• Provide direct conversion from LocalDate and LocalDateTime to java.util.Date
• More constructors and methods for Duration
• More constructors for DateTime
• Support JDK 1.6 pluggable resource providers
• More compatible with the Java Memory Model in JDK 1.5 and higher
• Added support for Joda-Convert without adding a dependency

Hopefully, you can just drop Joda-Time into your application and you will not notice any change. However, you should definitely read the release notes first.

Finally, as I'm sure someone will ask, this doesn't affect ThreeTen/JSR-310. This release was necessary to finally release a lot of bugs and enhancements in Joda-Time.

PS. As a bonus, there is a re-release of the JSP tags library, as apparently it wasn't in maven central correctly up until now.

PPS. For Hibernate 4.0 support, please use the usertype project.

Sunday July 24, 2011

Reversed type declarations

I can't write about Kotlin without first talking about the folly of "reversed" type declarations. Sadly this disease has afflicted Scala, Gosu and now Kotlin (but perhaps its not yet too late for Kotlin ;-).

Reversed type declarations

When I see a new language, one of the first things I look at is the parameters and variable declarations. For this blog I'll refer to them as "standard" (like Java, Ceylon and Fantom) and "reversed" (like Pascal, Scala, Gosu and Kotlin).

  // standard
  Type variableName
  
  // reversed
  variableName : Type

Here I compare a Java and Kotlin method, although the principle is similar for Gosu, Scala and quite a few others.

  // Java
  public void process(String str) { ... }
  
  // Reversed lang (Kotlin or any similar language like Scala or Gosu)
  fun process(str : String) { ... }

When I see the latter, I cringe. Its usually a sign that the language isn't going to win me over.

So why the big deal?

For a start, there is one extra syntax character, the colon. Given that this is unnecessary (as shown by Java), it continues to be surprising to me that languages that aim to be less verbose than Java begin with something that is more verbose.

Ignoring the annoyance of having to type the extra character, the two examples above are still fundamentally readable. The human eye (specifically my human eye, but hey I'm being opinionated...) can cope with the extra colon and "reversed" declaration order. However, add complexity and the situation changes:

  // Java
  public void process(String str, int total, List<String> input) { ... }

  // Reversed lang
  fun process(str : String, total : int, input : List<String>) { ... }

As more parameters are added, the eye has more difficulty in picking out where one parameter ends and the next one starts. This is simply because the colon is more visually arresting than the comma, so as the eye scans the line it breaks up the parameters using the colons, not the commas. Thus at a glance I see "str", "String, total", "int, input", "List<String>". In fact, my eye sometimes doesn't see the commas at all, thus I get "str", "String total", "int input", "List<String>" which is horribly broken.

In order to actually read the information, I have to slow down and take longer. But when designing a programming language, it is rapid and quick readability that matters. Slowing me down on reading is a Bad Thing. So, beyond being unnecessary, the extra colons are actually making the code significantly harder to read (and write!).

But it gets worse:

  // Java
  public Future<Person> process(String str, List<String> input) { ... }

  // Reversed lang
  fun process(str : String, input : List<String>) : Future<Person> { ... }

Now, we have a return type, again separated by a colon. The use of the same character (yes that is another verbosity character I have to type) makes it especially difficult to visually parse. For me, the strength of the colon overrides the end bracket, thus I end up seeing "Future<Person>" as a parameter. Effectively my eye is parsing the line in a fraction of a second, but it gets to the end and has to double back to "push" the last thing it saw onto the "return type" stack. Try this one if you're struggling to see the issue. Note how the types and colons dominate and flow into one another, causing the distinctions as to their meaning (type of a parameter vs return type) to be lost:

  fun process(a : Int, b : Int, c : Int) : Int { ... }

As an aside, lets look at default parameters, which many new languages support (using an equivalent syntax for Java):

  // Java
  public Future<Person> process(String str, List<String> input, int total = 0) { ... }

  // Reversed lang
  fun process(str : String, input : List<String>, total : Int = 0) : Future<Person> { ... }

Now it is really broken! Now I've got "Int = 0" staring me in the eye, which really is not what the programmer was trying to express. Again, that visual barrier of the colon, together with the type, makes it very hard to connect the actual parameter name "total" with the value it has "0".

The real test for the syntax is the more complex case of higher order functions. This varies a lot by language, so lots of examples (hopefully accurate - I don't have time for lots of testing). I'm simulating some syntax for Java and Fantom:

  // Java
  public <T, R> List<R> transform(List<T> list, Transformer<T, R> transformer) { ... }
  public <T, R> List<R> transform(List<T> list, #R(T) transformer) { ... }  // lambda strawman
  public <T, R> List<R> transform(List<T> list, {T => R} transformer) { ... }  // BGGA
  
  // Ceylon
  shared List<R> transform<T, R>(List<T> list, R transformer(T)) { ... }
  
  // Fantom
  R[] transform<T, R>(T[] list, |T -> R| transformer) { ... }  // simulated syntax

  // Gosu
  function transform<T, R>(list : List<T>, transformer(T) : R) : List<R> { ... }
  
  // Kotlin
  fun <T, R> transform(list : List<T>, transformer : fun(T) : R) : List<R> { ... }
  
  // Scala
  def transform[T, R](list : List[T], transformer: T => R) : List[R]

The Java strawman and BGGA examples and Fantom pseudo-example demonstrate that a form can be created where higher order declarations are possible using "standard" declarations. Ceylon chooses to go down the C style route, mixing the parameter name in the middle of the return type and arguments. I don't find Ceylon's choice as readable to my eye when scanning as the Strawman/BGGA/Fantom pseudo-examples because it mixes the type and the variable name.

The three "reversed" declaration approaches are very different. Gosu (if I've read the documentation correctly) makes a very weird choice as the element after the colon is the return type of the function not the type of the variable "transformer" within the method as would be expected most of the time. Kotlin's choice is also poor, as it now means there are two colons in the parameter declaration, one to separate the variable name from the type, and one to separate the function type input from its output. Scala's is the most rational of the "reversed" declaration styles. However, I find the lack of anything surrounding the "T => R" means that the eye struggles to find the start and end of the type in more complex examples, which is essential to finding the variable name.

Of these, the Strawman/BGGA/Fantom pseudo-examples and most readable of the first group and Scala in the second group (ignoring the real Java example for a minute, and noting that Scala would be clearer with something around the function type). That is because when I'm performing the eye parsing/scanning I've been talking about, I am essentially trying to grasp the signature. To do that I need to know the number of parameters, their names and their types. Specifically, I want to put the name and type into different mental boxes. Mixing the name and type as Ceylon and Gosu do makes that harder, while Kotlin's additional colon simply creates another fence for my eye to have to jump.

To do this full justice, I should really have some examples of function types of function types. However this blog is already very long...

Finally, I'll point out that this isn't just confined to method parameters, but also to variable declarations such as local variables. Again, this gets complicated between language in the detail, so I'll compare to a typical "reversed" type language using a braindead stupid example:

  // Java
  private String process(int total) {
    String str = Integer.toString(total);
    return str;
  }

  // Reversed lang
  fun process(total : Int) : String {
    val str : String = Integer.toString(total)
    return str
  }

Again, the "reversed" example is telling me that "String = ...", not "str = ...". In logical terms, its utterly broken.

OK, OK, I can hear you yelling "type inference":

  // Java
  private String process(int total) {
    val str = Integer.toString(total);
    return str;
  }

  // Reversed lang
  fun process(total : Int) : String {
    val str = Integer.toString(total)
    return str
  }

So, I cheated right? By inventing a Java type inference syntax. Well, I'm making the point that type inference need not be limited to new languages, Java or any language using the "standard" type declaration style can have it too (and Fantom and Ceylon do). Thus, we should judge the variable declarations by the long form, even if it is not used for local variables all the time. And as shown above, the long form is awful in the "reversed" style. I am most emphatically not assigning the total to "String", I'm assigning it to "str", and that is what the code should say.

I'm sure if you've read this far you have a number of comments. Perhaps you believe tooling solves the issue, maybe syntax colouring? Well, I'll simply say that while tooling helps, you should still be able to understand the language without it, even if just for command line diffs around a version control system. Or perhaps you're objecting to my methodology of trying to visually parse a line in a glance? Its how I work, don't you do scan code too?

Let me be clear, in none of the above do I mention the task of the compiler. My sole focus is on the developer reading the code, and an order of magnitude less writing code.

Any argument in support of "reversed" type declarations should never be based on relevance to the compiler or some other element of type theory.

My view is that the usability to the mainstream developer is what matters. And that is primarily about ease of reading what is written. I have endeavoured to show that the reverse style hampers readability, and is unnecessary to achieve the same goals of a more complex type system that are sometimes used as justification.

Summary

I'm arguing that the "reversed" type declaration style is flat out harder to visually parse, and should therefore be rejected by language authors, even if they believe they have sound compiler or type theory rationales. Programming languages exist primarily for developers, not to aid the compiler or underlying theories. "Write once, Read many" must be the first law of language design!

I am thus hugely disappointed that Kotlin, which has many fine features taken from Fantom, did not think through this choice in more detail, and I plead with the authors to change their minds before Kotlin is locked down.

Opinions welcome, and I'm sure there will be lots...

Thursday July 21, 2011

Kotlin and the search for a better Java

Another day, another new JVM language. This time Kotlin.

My first point here is that another group is willing to go public and say that Scala is too complex. It is easy to miss this, but anyone writing a new language right now (Kotlin, Ceylon, Gosu, Fantom, ...) is implicitly saying Scala isn't right. Of course I don't expect Scala supporters to like this or agree with it, but the truth is that I and many others have looked at it and run fast in the opposite direction.

As I've commented before, my dislike for Scala is not support for Java. There is a lot wrong with Java, and that cannot be sorted out without breaking backwards compatibility. Elements like primitives, arrays, wildcard generics and basic operators like equals. That is why I have proposed a backwards incompatible Java - #bijava.

More generally, my position is that as a community there is a role for a popular, statically typed, industrial, Java-like langauge without Java's warts. The Java-like also means a design that manages complexity and is usable by the mass-market. Fantom, Gosu, Ceylon and Kotlin are targetting that market. Groovy, Clojure, Scala and many others are simply not targetting that specific market.

I can't comment that much on Ceylon as it is more vapour than reality and from the little public information appears to have some dubious design decisions, especially around verbose words rather than syntax. I've also not studied Gosu much for some reason, yet in a day I've looked at Kotlin a lot (not sure why Gosu doesn't excite me...).

Fantom is probably the most different of the four. It provides a new platform, which happens to run on the JVM (and in the browser via Javascript!). The core Fantom library is new and dedicated to Fantom, with some different design principles to those of Java. It also runs its own form of bytecode, allowing deep immutability, non-null types, full modules and reified generics amongst other things. In fact, one could question whether Fantom fits into the group of four at all, however it does fit the criteria of statically typed, industrial and Java-like.

So, what about Kotlin? Well at first glance it gets a lot right, starting with null-safety and type inference. However, I have real issues with some features, which I think a proper Kotlin blog piece should focus on.

More generally if I could wave my magic wand, I would probably wish that Ceylon and Kotlin would merge into a single project. (Gosu and Fantom are now used in production, and Fantom has many different goals, so both are harder to change now.) Basically, we need the energy of all that input (and associated money) but with a better, single focus. Both Ceylon and Kotlin are still mostly paper languages, and both are trying to achieve the same thing (with Kotlin looking closer than Ceylon at this point). RedHat and JetBrains could you please have a conversation? (I'm happy to mediate if desired.)

Summary

I still like Fantom and I think most people hugely underestimate it, especially those from a Scala background. Fantom's rethinks what a language should be from the platform/productivity perspective with deep immutability, deep modules, no shared state and a practical type system that aims to eliminate bugs not be in absolute control. Scala is in many ways, light years behind Fantom.

And this is my key point with Kotlin too. Simply focussing on syntax is worthy, but kind of misses the point. Syntax exists simply to express the programmers intent in a way that should be readable years later. What makes a bigger difference are the productivity issues that are not typically thought of when talking about a language - versioning of code, which logging library to use, how to access configuration or injected state. Kotlin tackles the syntax parts pretty well, though not perfectly. But its not clear yet if they can grasp just how unimportant the syntax is relative to language related productivity gains.

Thursday June 09, 2011

JCP - Bonn meeting October 2010

In writing my last blog entry I realised that I never wrote up details of the Executive Committee meeting in October 2010. Thus, this post is is mostly for the historical record, but it could be a useful pointer for the Google/Oracle lawsuit.

Why is this still relevant?

The Apache Harmony case can seem like an odd one to write about today. After all aren't we all happy that Oracle is investing in Java and Java SE 7 will be released soon? But in achieving this there has been collateral damage that cannot be resolved until the Oracle Gooogle lawsuit is resolved.

Apache only joined the JCP once it had established the right for any JSR to be implementable by Open Source in general, and Apache in particular. When Sun signed up to the JSPA legal agreement, they knew they were signing up to something that would allow another implementation of the Java SE specification.

When Apache Harmony began, Sun said nothing about restrictions that would prevent it completing. Only when Apache asked for the testing kit did Sun say something (or as I would say, change the rules).

The timeline is important. Legally important. In May 2005 we have this:

Part of the reason these Apache projects are possible is that the JCP has been working over the last few years to clarify the role of open source projects and also to improve overall transparency. For example there were changes to the intellectual property rules in JCP 2.5 specifically designed to allow open source implementations. There have also been a number of transparency improvements in JCP 2.6 to allow earlier access to specification information. In addition to those JCP updates, Sun created a scholarship program under which we have provided Apache (and other not-for-profit groups) with free access and free support for Sun's Java compatibility test suites.
...
The licensing rules for J2SE 5.0 were carefully designed to allow independent, compatible open-source implementations of the J2SE specification. Personally, I am not entirely sure if the world really needs a second J2SE implementation, but at the same time I am also glad to see that all the effort we put into getting the rules and the licensing issues straightened out is actually proving useful!
Graham Hamilton's blog VP and Fellow at Sun responsible for Java at the time, written May 7, 2005.
(My emphasis)

And lets look at how that was reiterated in the business terms for Java SE 6:

JSR 270: JavaTM SE 6 Release Contents
...
2006.10.24:
2.18 Please provide a description of the business terms for the Specification, RI and TCK that will apply when this JSR is final.
...
7. Nothing in the licensing terms will prevent open source projects from creating and distributing their own compatible open source implementations of Java SE 6, using standard open source licenses. (Yes, you can create your own open source implementation of Java SE 6 if you really want to. But we're also doing everything we can to make it easy for you to use the original RI sources! See http://jdk6.dev.java.net.)

JSR-270
(My emphasis)

These two passages create a clear intent that Sun would not do anything to block Apache Harmony as an open source project under the Apache license. With the JSPA legal agreemnet and these additional statements, Apache acted, only to later be blocked.

I said "Legally important", because there is probably an element of estoppel here (but IANAL!).

Basically estoppel prevents a party (Sun) from promising one thing, causing another party (Apache) to act on the promise, and then changing its (Sun's) mind to its own advantage. Its still legally important today because a key plank of Google's defense against Oracle is that Apache Harmony would be a valid implementation of Java. To balance this, I must point out that Google wasn't directly involved in the Apache-Oracle battle (its a third party), so estoppel would only apply if Google could justify it acted based on the same promises (remeber IANAL!).

5th/6th October 2010 face to face JCP Executive Committee meeting

So, back to the JCP EC meeting. This meeting was the key event where Oracle announced its final decision on Apache Harmony. This was before Doug Lea, Tim Peierls and the Apache Software Foundation left the JCP, and before IBM moved from Harmony to OpenJDK.

The JCP had been debating the issue since 2007 (with private efforts to resolve the situation being made by Apache and Sun before it came to the JCP and before it became public). Oracle had originally (with everyone else) sided with Apache. Until Oracle took over Sun.

The first part of the meeting minutes show a vote:

Prior to this meeting Doug Lea proposed and Tim Peierls seconded the following motion:

It is the sense of the Executive Committee that the JCP become an open independent vendor-neutral Standards Organization where all members participate on a level playing field with the following characteristics:

* Members fund development and management expenses.
* A legal entity with by-laws, governing body, membership, etc.
* A new, simplified IPR Policy that permits the broadest number of implementations.
* Stringent compatibility requirements.
* Dedicated to promoting the Java platform.

Furthermore, the EC shall put a plan in place to make such transition as soon as practical with minimal disruption to the Java Community.

* Apache – yes
* Credit Suisse – yes
* Eclipse – yes
* Ericsson – yes
* Fujitsu – abstain
* Google – yes
* HP – abstain
* IBM – yes
* Intel – yes
* Werner Keil – yes
* Doug Lea – yes
* Oracle – did not vote
* Tim Peierls – yes
* RedHat – yes
* SAP – yes
* VMWare – yes
The motion was therefore carried.
EC meeting minutes

Clearly, most EC members viewed the correct solution to be an independent JCP. We also note two of the "quieter" JCP members avoiding voting Yes (HP and Fujitsu).

Patrick reported that Steven Chin replaces Wayne Carr as Intel's primary rep
EC meeting minutes

Wayne Carr was a strong supporter of Apache. He was replaced at the key moment in the debate, right before the key (first) vote on Java SE 7. I'll leave readers to speculate on why that change was made.

Don Deutsch reported that Oracle has spent several months in consultation with EC members, with developers, and with Java customers trying to work out a way forward. They have looked at a wide range of options and discussed various working arrangements, and finally concluded that Apache Harmony sets the stage for an inevitable fork. Even if Apache Harmony itself strictly complied with the TCK there is no ability to enforce compliance downstream (Apache cannot guarantee that either Apache communities or downstream users will strictly comply with compatibility requirements.) Consequently, Oracle has concluded that they cannot grant Apache a TCK license. Don stated that this decision is final - it will never change. He said that Oracle doesn't believe that the Java community fully understands the risks and consequences of a fork, noting that the Google Android situation is analogous to that which caused Sun to take legal action against Microsoft several years ago.
EC meeting minutes (my emphasis)

This is the key paragraph. The argument is couched in terms of a fork and references Android directly. No mention is given of Java EE where competing implementations are deemed positive, not negative. More specifically, Oracle state that their decision is final. Personally, I disagree with the notion that the Android case is analagous to the Microsoft case.

Don acknowledged that the community is frustrated by the lack of technical progress caused by this licensing dispute and said that Oracle plans to move forward with delivering new Java technologies through OpenJDK. Developer reaction to the plans announced at JavaOne was very positive. He said that Oracle will submit umbrella JSRs for Java SE 7 and Java SE 8 shortly after this meeting and that Mark Reinhold will present details of their contents later today. The JSRs will be presented with similar license terms as previous umbrella JSRs.

Don stated that the developer community did not want a licensing dispute to halt the evolution of the Java platform, and argued that it was in the interests of both Oracle and the community to move forward. Oracle very much wanted to make progress within the JCP, but if this was not possible it would be necessary to find another way to advance the platform.

Don concluded that it is in Oracle’s interest to have partners who increase their use of Java. He said that Oracle wanted to make OpenJDK more collaborative and more useful to licensees, since other Java vendors have a vested interest in moving forward with Oracle. He said that all partners are welcome to join Oracle, and emphasized that Oracle wants to enter into long-term and stable contracts with licensees to reduce the risks to both parties. He promised that Oracle will be a trusted and stable Java partner and reiterated that Oracle's primary economic interest is in furthering Java rather than in extracting revenue from it.
EC meeting minutes (my emphasis)

The second of these paragraphs indicates that there was a clear statement that Oracle would proceed with Java SE 7 with or without the blessing of the JCP. (Apparantly it was stated more clearly in person)

As stated, the JSRs for 7 and 8 were released a few days later, and contained the same restrictions.

The last paragraph is interesting when you look at timing again. The paragraph provides a welcome to other partners to join OpenJDK. An offer made on the 5th October. One week later on the 11th October, IBM abandoned Harmony and joined OpenJDK. Would the Oracle and IBM reps at that table have known about that upcoming deal?

Jason Gartner said that IBM was disappointed with this decision - it wasn't what they wanted. He expressed the hope that if he was in Don's position he would be giving a different message, but of course he was not. He argued that Oracle is not Sun, and will not change their minds. The important question was what to do next. He pointed out that the JCP had been in stalemate since 2007, and it was necessary to be constructive and to move forward. He said that he was looking forward to hearing about the plans for Java SE 7 and SE 8.
EC meeting minutes

Would IBM therefore have been a better steward of Java? IBM believe so, perhaps unsuprisingly. But this also suggests that IBM is looking forward, beyond the dispute.

Scott Jameson said that the topic under discussion was licensing and Field of Use (FOU). He noted that FOU restrictions were the root cause of the dispute, and asked Don to comment on whether or not FOU restrictions violate the JSPA. Don responded that Oracle intended to continue to offer licenses on the existing terms.

Doug Lea pointed out that there is no Java 7 JSR because the EC believes that members should conform to JSPA. He argued that FOU-restricted licenses violate the JSPA and suggested that the JSPA be changed. Josh pointed out that Oracle is on record as saying that FOU restrictions violate the JSPA. He asked whether they still feel that way. Ken Glueck responded that Oracle came to talk about the Apache issue. He pointed out that Don had been straightforward and candid. Doug responded that EC members accept Oracle's determination not to grant Apache a license, but argued that they cannot vote to approve a violation of the JSPA. He argued that if Oracle wished to withhold a license from Apache it must be on different grounds or the JSPA must be changed.

Mike DeNicola asked whether Oracle was saying that they wouldn't grant a license to Apache under any circumstances or whether - like Sun - they ware saying that they wouldn't grant a license except with FOU restrictions. Ken Glueck confirmed that Oracle's position is the same as Sun's. Geir Magnusson said that this was the impression that he also had. Don Deutsch confirmed Oracle's position was that no unencumbered license would be offered to Apache.

Doug asked Oracle to acknowledge that it was asking the ECs to condone breaking the JSPA rules. He said that if he was put in a position where he had to condone breaking the rules he would have to resign. Ken said that he understood, and would regret it if Doug resigned, but pointed out the importance of allowing the conversation to go forward. Josh Bloch asked again whether Oracle, who were on record as saying that to deny Apache a license without FOU restrictions is a violation of the JSPA, still feel that way. Ken Glueck responded that Oracle were not prepared to answer a legal question. Josh responded that Oracle had been willing to vote on this matter twice in the past. Ken responded that this is the situation now. Tim Peierls noted that Oracle has had plenty of time to prepare an answer. Ken pointed out again that the platform is stuck and we need to move forward.

Doug Lea suggested that Oracle propose JSPA changes to legalize the situation, making it possible for ECs to approve the SE JSRs since they would no longer be in violation of the JSPA. He argued that this would be a simple matter, and should be done as part of the JCP.next proposal. The JSPA could be changed so that Oracle is no longer obliged to grant Apache a license. Ken thanked Doug for his constructive suggestion, noting that the JCP.next discussion hadn't yet happened. He asked others for their thoughts on this matter. Several members pointed out that modifying the JSPA was a more complex task than Doug had suggested, and might take months or even years.
EC meeting minutes (my emphasis)

In the first highlighted part, Doug Lea links Java SE 7 to the dispute, confirming my original blog in 2009.

In the rest of this passage, Oracle effectively blocks questions about why they changed their mind on the JSPA. In the second and third highlighted parts, Josh Bloch pushes the point - Oracle agreed that Sun's action was in violation of the JSPA, yet now they were doing the same. The point was batted away.

Personally I believe that Sun's reason for denying Apache Harmony was to protect Java ME licensing revenue (Sun was broke), while Oracle's reason was the Google lawsuit (which is clearly higher priority than a vibrant Java community). Corporately, Oracle must feel very strongly about attacking Google, because they were willing to change their minds on this issue in a way that looks incredibly stupid, especially with no attempt to justify it (because its at the heart of the lawsuit against Google...).

EC members also called for the JSPA to be changed as they could not condone breaking it. This proposal to change the JSPA is essentially from a similar vein as my Split JCP proposal. Doing so would have greatly lessened the damage to the JCP, but Oracle chose not to do so (perhaps because doing so would be a tacit admission of fault in a document key to the Google lawsuit).

Mike DeNicola addressed the question of why the EC was focusing on licensing given its supposed charter. He noted that the JSPA requires the submission of a JSR and that the ECs were the Expert Group for the JSR that defined the JSPA. They wrote the rules (he was involved.) This is how they got involved in licensing. When the FOU issue was first raised EC members said "when we wrote the current version of the JSPA we intended to move towards standardized open licenses. We removed restrictions from the spec license, but we overlooked the TCK license." Sun lawyers had argued that the JSPA didn't restrict what could be written in TCK licenses but EC members disagreed. Oracle's lawyers now seem to have reached the same conclusion as Sun did. He argued that it was necessary to move forward and that this discussion was not constructive. Lawyers should review the JSPA and if necessary they should suggest changes to clarify its intent.
EC meeting minutes (my emphasis)

In this later passage, we have some history on how the mess happened. It is suggested that the testing kit was an oversight and that the lawyers of EC members and Sun (now Oracle) disagreed over whether the JSPA did or did not allow testing kit restrictions.

But if you look at the statements at the top of this blog, you'll get a different set of opinions, suggesting Sun's legal opinion was a later thought to wiggle out of a contract they no longer liked.

Josh read the resolution recently proposed by Doug Lea (as recorded at the beginning of these minutes.). He noted that it was passed without objection, but without a vote from Oracle. He had asked why Oracle didn't vote. Don Deutsch asked why Josh was raising this again. Josh responded because he still hoped to see an independent organization. Don responded that he had originally proposed this motion back in 2007, and expressed his surprise that Josh cast Google's vote in favor of the motion since the wording included a statement about the importance of compatibility, which Google did not seem to care about. He pointed out that nobody had offered to pay for the Java assets, nor for the very significant ongoing expenses of developing the Java platforms. Oracle did not vote on the resolution because it was orthogonal to the issues the EC was addressing. Later in the discussion Josh responded that Google does care about compatibility - when they build something that they claim is standards-compliant they work hard to make it so.
EC meeting minutes

In this later revealing section, Google and Oracle clash. Oracle says "Google doesn't care about compatability". Google says "we do when we claim it is compatible" (ie. we didn't claim Android is compatible).

Don's comments about Java assets prompted a discussion about the difficulty of forming an independent organization without control of the Java brand and without ownership of the core assets of the Java platform. (Josh Bloch and Geir Magnusson argued that it would not be necessary for an independent organization to own the Java assets.) Jason Gartner pointed out that Oracle owns the Java brand and will not give it up. EC members could still work to influence Oracle. There was general agreement (if not approval) of this situation.
EC meeting minutes

And here is the final part of that section. An independent JCP is meaningless without the Java brand, but Oracle will not give it up. The final sentence rather suggests the EC members were beginning to give up, which they formally did when they passed the Java SE 7 JSR into being at the expense of Doug Lea, Tim Peiels and Apache leaving the JCP.

Summary

As I said, I'm no longer here to change minds. But as this blog effectively acts as a record of the dispute it only seems right to documnet a missing piece. (I didn't write about this at the time because these meeting minutes were, unsurprisingly, delayed in publication).

And I have pointed out, this dispute is clearly referenced by Google as part of their defence against Oracle.

Stephen Colebourne
Apache Software Foundation member, speaking personally
Oracle Java Champion, speaking personally
Not a committer on Harmony or OpenJDK

Tuesday June 07, 2011

Java SE 7 passes in the zombie JCP

So in the end, there is a "Java SE 7". The vote at the JCP Executive Committee passed. But since I first started following this story back in 2009 its been a long road with a huge political fallout.

The Yes vote today happened because Oracle made a decision to kill Apache Harmony. A decision that was a continuation of Sun's policy, but one that Sun was not strong enough to make happen. Oracle's decision was to tell the JCP that it would proceed with version 7 no matter what the Executive Committe voted.

The cost of this decision was that Doug Lea, Apache and Tim Peierls left the Executive Committe. As such, I view the JCP as a zombie and under no circustances an Open Standards body, as promised in 1996.

You cannot claim to be an Open Standards body if you do not allow implementations of the specification.

So, this vote today is realy the end of a very long story - see below for the full history of blogs. As always in the JCP votes, the Yes/No vote is only half the story. The comments tell you the rest:

On 2011-05-31 VMWare voted Yes with no comment.
------------------------------------------------------------------------------
On 2011-05-31 Oracle voted Yes with no comment.
------------------------------------------------------------------------------
On 2011-06-01 Eclipse Foundation, Inc voted Yes with no comment.
------------------------------------------------------------------------------
On 2011-06-01 Intel Corp. voted Yes with no comment.
------------------------------------------------------------------------------
On 2011-06-02 IBM voted Yes with the following comment:
IBM's vote is based on the technical merits of this JSR and is not a vote on the licensing terms. IBM supports licensing models that create an open and level playing field by allowing third parties to create independent implementations of Java Specifications and that do not allow individuals or companies to exercise unnecessary control for proprietary advantage. We support open source as a licensing model for contributions in the JCP, and would hope others will support this direction. This comment is not necessarily directed at the current business or license terms for this JSR, however, it is a statement of IBM's preferred licensing model.
------------------------------------------------------------------------------
On 2011-06-02 RedHat voted Yes with the following comment:
Red Hat's vote is based on the technical merits of this JSR and is not a vote on the licensing terms. Red Hat would prefer a licensing model that creates an open arena for everyone, including those not members of the JCP and removes any ability for one individual or vendor to exert undue control over a standard. We are an open source company and hence would like to see such a licensing model for JCP contributions. Note however, that this comment is not necessarily directed at the license terms for this JSR, but is a statement of Red Hat's preferred licensing model.
------------------------------------------------------------------------------
On 2011-06-02 SAP AG voted Yes with no comment.
------------------------------------------------------------------------------
On 2011-06-02 Google Inc. voted No with the following comment:
While Google supports the technical content of this JSR, we are voting no because of its licensing terms. As per the JCP resolutions of 9/25/2007 and 4/7/2009, "TCK licenses must not be used to discriminate against or restrict compatible implementations of Java specifications by including field-of-use restrictions on the tested implementations. Licenses containing such limitations do not meet the requirements of the JSPA, and violate the expectations of the Java community that JCP specs can be openly implemented."

The proposed license clearly violates this requirement (see Exhibit A, Section II). Oracle was duly reminded of this when JSR-336 was first proposed, but has done nothing to address the issue. It would be wrong to condone the inclusion of field-of-use restrictions in a TCK license, as this clearly violates the JSPA, by Oracle's own admission. Google does not want to slow the progress of this release, but we do believe it is critical that this issued be addressed, in order to comply with the JSPA and to preserve the openness of the Java platform.
------------------------------------------------------------------------------
On 2011-06-05 Keil, Werner voted Abstain with the following comment:
While I wish new and improved versions of Java to be released as soon as possible, given multiple delays in the past, the lack of transparency both in this Umbrella JSR and relevant components (especially Project Coin) fuel my decision to abstain here.

Not only have other EC members expressed their discomfort with this situation and confirmed some of these developments behind closed doors they demanded being opened, haven't been. The Spec Lead has also publicly admitted he wished, more of the SE-related activities were open and transparent than they were so far.

Given the efforts for more transparency by JSR 348 were just accepted by a larger majority than even this JSR or most others in SE/EE, I hope some more existing JSRs respect this and become more open especially if related projects even use names like "Open..."
------------------------------------------------------------------------------
On 2011-06-05 London Java Community voted Yes with the following comment:
The LJC votes Yes to the JSR for SE 7 based on its technical merit and our very strong desire to get Java moving again.

We note that the archives for some of the Expert Groups that comprise this JSR have not yet been made public, despite a promise from Oracle to do so. We do not feel that this is appropriate for a public and open standards body. In particular, Oracle's silence as to why the archives have not been made public is harmful to community participation, i.e. the community has no access to historical technical discussions, which are vital for participating in future Java language initiatives.

Going forward, we are unlikely to support any JSRs that do not meet minimum standards of transparency.
------------------------------------------------------------------------------
On 2011-06-06 Hewlett-Packard voted Yes with no comment.
------------------------------------------------------------------------------
On 2011-06-06 Goldman Sachs & Co. voted Yes with the following comment:
Goldman Sachs votes Yes on JSR 336 based on its overall technical merit and our support for moving the platform forward

This umbrella JSR suffers from an overall lack of transparency compounded through inclusion of other JSR's as components which in turn are opaque. We clearly need to improve the transparency of the constituent parts and ensuing that the community has a clear understanding of what is coming. We are hopeful that JSR 348 (JCP.Next) starts to address these issues
------------------------------------------------------------------------------
On 2011-06-06 SouJava voted Yes with the following comment:
SouJava votes yes on this JSR based on its technical merits. Our members are satisfied with the evolution of the technology, but are unhappy with how the licensing terms in this proposal discriminates against open source implementations, and how this can negatively affect or influence other JSRs. We also have concerns about the lack of transparency of some of the JSRs involved in JSR 336, and how it has negatively impacted the Public Review process. We urge the Spec Lead to rectify this immediately, long before the Final Draft is presented.
------------------------------------------------------------------------------
On 2011-06-06 Ericsson AB voted Yes with no comment.
------------------------------------------------------------------------------
On 2011-06-06 Fujitsu Limited voted Yes with the following comment:
Fujitsu's vote is based on the technical merits of this JSR, but not based on the licensing terms.

------------------------------------------------------------------------------

Considering this is a major release of Java, a platform used by more than 8 million developers, those comments are not a ringing endorsement.

Specifically, the comments note that Oracle's promises to have open and transparant JSRs have not been fulfilled.

More broken promise then. Why am I not surprised.

As I said, there used to be a Deal. Where the owner of Java makes a large investment and the community makes it relevant. By constantly thumbing its nose at the Java community, Oracle is harming its own interests. But I don't think they care anymore.

And yet, it could still be fixed - <tounge-in-cheek>Apache OpenJDK anyone</tounge-in-cheek>?

Stephen Colebourne
Apache Software Foundation member, speaking personally
Oracle Java Champion, speaking personally
Not a committer on Harmony or OpenJDK

This is the history of my blogs on the topic. I've starred my favourite entries:
No more Java 7 * - March 2009 - the first article linking Java SE 7 and Apache Harmony
A question of IP - March 2009 - a decription of how IP works in the JCP
Sun, Apache & IP - in pictures! * - March 2009 - a pictorial view of how IP works in the JCP
Shedding new light on No Java SE 7 JSR * - March 2009 - the first analyis of the JCP Executive Committee meeting minutes
The ASF perspective - April 2009 - the first analysis of the Apache Software Foundation meeting minutes
The JCP doesn't exist * - April 2009 - showing how the JCP is a department, not an organization
The Oracle perspective - April 2009 - discussing what Oracle might do now they were taking over Sun
US DOJ investigation - June 2009 - pointing out that the DOJ could intervene
JSRs submitted over time * - October 2009 - graph of the submission of JSRs over time
Power corrupts. Absolute power corrupts absolutely. * - August 2010 - Oracle chose to sue Google and break the community
The end game - October 2010 - Oracle entices IBM from Harmony to OpenJDK
Does Oracle have enough votes - October 2010 - analysis of voting intentions in the first vote on Java SE 7
Pragmatism or bust * - October 2010 - voting Yes rewards the bully, but voting No would end the JCP
Split JCP proposal * - October 2010 - IMO, still the best approach for the future of the JCP
Stacking the JCP election - October 2010 - where I claimed Oracle was manipulating the JCP election
My election choices - October 2010 - my choices for the JCP Executive Committee election
Bablyon 5 & the Great War of Java * - October 2010 - a look at the big picture using Bablylon 5 as an analogy
Four viewpoints of War - October 2010 - analysis of the comments of four major players
JCP election result - November 2010 - summary of the JCP Executive Committee election
Premium JVM thoughts - November 2010 - a premium JVM scared people but proved to be a non event
ASF ready to leave the JCP - November 2010 - the ASF announces that the Java SE 7 vote could result in it leaving the JCP
Oracle replies to the ASF - November 2010 - I decode the Oracle press statement to mean the exact opposite of what it appears to say
Java SE 7 and 8 JSRs published * - November 2010 - detailed analysis of the licensing mess that Java SE works under
Devoxx whiteboard votes - November 2010 - developer votes referring to the issue
First Java SE 7/8 votes pass - December 2010 - brief analysis of the voting, noting Intel's change of view and representative
Is the JCP dead * - December 2010 - questions the viability of the JCP once Doug, Tim and Apache left
The Deal * - December 2010 - my take of the deal between Sun and the community that Oracle doesn't understand
What about JSR-310 - December 2010 - my choice to continue JSR-310, using the JCP as a tool rather than a standards body

Sunday May 29, 2011

Beans and properties

A few words on beans and properties in Java, prompted by the JavaFX 2.0 API and my work on Joda-Beans.

Beans and Properties

I was prompted to write this by the new API of JavaFX 2.0. It has a new extension to JavaBeans that adds support for properties, so that the JavaFX GUI can bind one property to another. I've been writing property extensions to JavaBeans for over 12 years now, so I feel I should comment on the design.

The basic design approach is to have an object representing each property on a bean. This object allows meta-data attributes to be stored per property (useful for marking errors), property change listeners (for GUIs) and tool access without reflection (allowing XML input/output, XPath, JSON etc).

The basic design adds an additional method to the JavaBean pattern. Alongside getSurname() and setSurname() there would be the property method surname(). This would return a Property object, which supports querying and setting the value, checking whether it was read-only or read-write, finding its type and adding property change listeners.

  // basic querying via JavaBeans or property
  String surname = person.getSurname();
  String surname = person.surname().get();
  
  // additional features available with properties
  Property<String> property1 = person.surname();
  Property<String> property2 = person.property("surname");
  Map<String, Property<?>> propertyMap = person.propertyMap();
  person.surname().setAttribute("error", "MANDATORY");
  person.surname().addListener(...);

All the versions of this I've written follow a pattern relatively similar to that above. And the JavaFX 2.0 design is remarkably similar to this. However, some implementations of the pattern are more successful than others.

Version 1 - Original Joda

My initial work in the area was private, but I wrote a variation in open source around 2000 (unmaintained from 2002). This was the first Joda project (before Joda-Time), and still lives on the web here (Javadoc). Here is the original Joda approach (shown with generics for clarity):

  public interface Person {
    Property<String> surname();
    String getSurname();
    void setSurname(String surname);
  }

In that project, the class was created using a Proxy object. Clearly, this approach was rather limiting, as a factory had to be used to created an instance of a bean. For various reasons, including the factories, this version was abandoned.

Version 2 - Instantiated Property objects

This version existed at one of my old day jobs. It used code generation rather than Proxy, which was better. However, it used this approach to the beans:

  public class Person extends Bean {
    private final Property<String> surname = new Property(String.class);  // object created when person created
	
    public Property<String> surname() { return surname; }
    public String getSurname() { return (Strung) surname.get(); }     // state held on the property
    public void setSurname(String surname) { surname.set(surname); }
  }

Note that every time a Person is created, a Property object is created. Now imagine what happens when you have 20 properties on the bean - thats 21 object creations for an empty Person! And 21 garbage collections! It turns out that this design works OK for client side work, where there is a single machine and processing is limited by what one human can do. However, on the server side it performs horribly. Thus, it got converted into version 3.

Version 3 - Data on the bean

The version 2 approach stored all the state on the Property, but this approach does not work well with the JVM (it could, if the JVM/Java had the ability to inline the state of immediate child objects into the state of the parent object). Thus, the next approach was to move all the state from the property to the bean:

  public class Bean {
    private final Map<String, String> attributes;
  }
  public class Person extends Bean {
    private final String surname;
	
    // code generated:
    public Property surname() { return new Property<String>(this, "surname"); }
    public String getSurname() { return surname; }
    public void setSurname(String surname) { this.surname = surname; }
	
    protected Object propertyGet(String propertyName) {
      switch (propertyName.hashCode()) {
        case 36527358: return getSurname();  // number is "surname".hashCode()
      }
      return null;
    }
  }

This approach moves all the state from the property back to the bean itself. The superclass would store any common items, like the attribute map (for error code) or the listeners, while the main person class stores the data using normal fields. The code generation adds a method that connects up the field without reflection.

Since the property is esentially stateless, it can be created on demand, used and then discarded. Short lived objects like that are good for the garbage collector, so long as they are not used too frequently. This design was successful on the server side, and supported all the other goodies, like conversion to and from XML and XPath searching, all without reflection. I believe that a dynamic code generation approach would also work, but the static code generation was effective enough that we didn't go further.

Version 4 - Joda-Beans (the next generation...)

The new Joda-Beans project (Javadoc) is an open source version of the version 3 approach, written from the ground up, but focussed on the essential elements for simplicity. It is currently in use at OpenGamma.

Again, a code generator is used to create the getters, setters and property methods, plus the supporting methods like propertyGet() and propertySet(). This time however, there is a greater focus on the meta-property level, which is the representation of the surname property as a concept, rather than tied to a particular bean instance. Think of the relationship between a property and a meta-property as similar to the relationship between an object and a class. (Effectively, Joda-beans defines both type 1 and type 2 properties.)

Rather than show a code sample here, I encourage you to look at some example classes to see what the code generator produces: Person, Address and the subclass CompanyAddress. The developer only writes the part outside the "code generated" block. The block is generated and re-generated without destroying any additional methods that are hand-written, so you can treat it as a "normal" class in all other ways.

Once again, the property objects hold no long-lived state and are created on demand. After 12 years of working with properties, I'm convinced that is the only design that scales acceptably for the server side on the JVM.

While the Joda-Beans project is not at v1.0 yet, it does work and enhances coding in Java. I'd love to have some early adopters try it out!

JavaFX 2.0

The JavaFX 2.0 approach uses stateful properties (Javadoc - javafx.beans packages). As I've discussed above, such an approach is powerful and conceptually simple, but has performance issues on anything other than the client side. Some articles on the JavaFX 2.0 approach are here and here Clearly, a major concern of the design is listeners, but I still have concerns about this approach.

Other implementations

There are other projects too that have tackled this problem, including Proper properties, Bean-properties, Better beans binding, and Remi's compiler changes (I'll add anything in the comments to this list). No project has won a large mind share though, and most seem unmaintained. A related solution is the ugly JPA typesafe extension.

Language solutions

I've said this before and I'll say it again. The lack of properties in Java is the biggest missing langauge feature and the amount of wasted development time globally by developers writing beans or tools to access properties is gigantic. Moreover, I believe that adding properties to Java is more important than both generics and closures, and should have been prioritised accordingly. (Every enterprise developer knows how much they use the broken JavaBean approach.)

The new JVM languages all have properties (no surprise there), although not all have proper support for obtaining an object to represent the property. Maybe one day, Java itself will support properties as a language feature, it is being vaguely mentioned now, but I'm not holding my breath.

Summary

Properties are a hugely beneficial language feature and raise the abstraction level of coding. While the Java language continues to not have them, you can use or help the Joda-Beans project to add them to your codebase.

My experience suggests that the JVM is poor at handling stateful property objects. Unfortunately, this is the route that JavaFX 2.0 has chosen. It might work for them (because they are client side, and because JVMs are getting faster), but I would advise against using JavaFX 2.0 properties on the server side without lots of performance testing.

Feedback welcome...

Thursday December 30, 2010

What about JSR-310?

So, if the Java Community Process is now a zombie Java Corporate Partnership, what should I do with JSR-310?

JSR-310

I've been highly critical of Oracle's stewardship of Java, particularly on the topic of the JCP. Choosing to U-turn on their previous public statements and force through the Java SE 7 and 8 JSRs was bad for Java and bad for Oracle. By accepting Oracle's actions, the remaining JCP executive committee members ended the notion of the JCP as an open standards body.

So, the question arises as to whether I should cut my ties with the JCP and terminate JSR-310.

The reasons to abandon the JSR are quite compelling:

• The JCP is no longer an open standards body.
• JSRs now exist that are in incompatible with the JSPA legal agreement.
• The Executive Committee has shown itself to be more willing to do Oracle's bidding than to hold them to account.

However, I must also ask what the result would be if I stopped the JSR:

• I would break a commitment I made to the community to improve dates and times in Jave SE.
• Someone else might take over the project, perhaps causing another poor Java SE date and time library.
• The majority of developers, those that don't read blogs and news sites, don't care about the corporate politics and would suffer without ever knowing why.

Ever been between a rock and a hard place?

The upside is that the process of the JCP provides all the power to the specification lead. So, since I, and Michael Nascimento, are the specification leads for JSR-310, we have all the power :-) Consider what I said when the JCP asked me about transparency:

Anyone can access these javadoc files by just clicking a link on the JSR 310 website at java.net. Stephen says, "We always intended to run the JSR as an open source project. That naturally means public code, and feedback. Collecting the feedback is easier if people can access it by the method they are most familiar with -- javadoc." Some Spec Leads may be reluctant to run their projects as openly as this, but Stephen has no qualms at all. He says, "It's often thought that being open will increase the mails and issues you have to deal with. The reality is that you tend to have to address the real issues earlier than you might otherwise have to, and that is a very good thing."

The key phrase is "run the JSR as an open source project". Because if the JSR is an open source project then the status of the JCP is more of a detail, an annoyance.

So, while my heart might tell me to walk away from the zombie JCP, my head tells me that I really don't need to. This is just an open source project which the community badly needs. And the JCP is just a tool I can use to access Java SE 8. I dislike maven as a tool, yet I still use it - I can dislike the JCP and still use it in exactly the same way.

Therefore, I've decided to continue work on JSR-310, targetting Java SE 8.

I've also taken advantage of the impending destruction of the original java.net website to move the code and discussion to sourceforge, where I hold most of my other projects.

I've also chosen to give the reference implementation a real name - ThreeTen. So, the ThreeTen project is the reference implementation of JSR-310. Development of both the reference implementation and the JSR will occur at the ThreeTen project.

• ThreeTen home page
• ThreeTen project page
• Develop mailing list

As part of this process I've attempted to clarify the legal position of contributions to the project, including discussion, without getting too legal. The concept is that the overriding principle is that the project works on trust and respect for the public good. Beyond that, version controlled contributions continue to use the BSD 3-clause license, and version controlled contributions destined for JSR-310 continue to require a contributor agreement. As such, existing mailing list subscribers need to resubscribe to the new mailing list.

I also leave open the option of writing a ThreeTen specification separate to the JSR-310 specification.

The aim of a ThreeTen specification would be to provide common terminology and behaviour at a higher level with the intention of sharing the knowledge gained to other areas of computing. This might be another language, library or specification, such as SQL or XML.

Finally, I'm pleased to confirm that I will continue to spend 20% of my work hours on ThreeTen/JSR-310 thanks to my employer OpenGamma where the project is a core part of their platform for financial analytics and risk management.

Stephen Colebourne
Project lead, ThreeTen
Co-spec lead, JSR-310

Monday December 20, 2010

The Deal

This is the story of a deal. Not one that was ever signed on a piece of paper. But one that was still very important. Its the deal between the owner of Java and the Community.

The Deal

This is my view about how the owner of Java and the Community have interacted:

The owner of Java makes a large investment.
The community makes it relevant.

By "owner of Java", I mean Sun, then Oracle.
By large investment, I mean money, development time, marketing and energy.
By relevant, I mean interesting and productive for mass use.

The key point is that this is a symbiotic relationship. The investment by the owner is worth much, much less if the language is not relevant. And the community needs an actively developed core and common set of rules to build upon.

When you consider the limited nature of the core of Java, this should be self evident. The core consists of the JVM, compiler, language and core libraries. Enough to get everyone started, but not that interesting in and of itself.

What is interesting about "Java" is everything else:

Servlets, JMS, Tomcat, JBoss, Lucene, JMX, Eclipse, Ant, Portlets, Lombok, Devoxx, Javalobby, Axis, JIRA, RESTEasy, Terracotta, Ivy, JSP, TestNG, Grails, Mule, Android, ICU4J, MyFaces, Scala, James, Geronimo, JCS, OFBiz, Jetty, GWT, Websphere, JFreeChart, JavaMail, FastUtil, Xerces, JDBC, Griffon, JProbe, SLF4J, Wicket, XOM, JavaOne, Seam, Emma, HttpClient, EHCache, TheServerSide, Roo, Mockito, HSQL, Guice, FOP, Kindle, Velocity, Clojure, JNDI, Clover, Hadoop, JSF, Jackrabbit, Livescribe pen, Commons, Hibernate, EJB, Tobago, IntelliJ, Jersey, Scalaz, HornetQ, JAX-RS, Lift, Derby, JUnit, Freemarker, JavaME, Mylyn, Gaelyk, MINA, Play, JBPM, Cobertura, Antlr, Artima, Findbugs, Hessian, OGNL, Quartz, Trove, Tales, Javolution, Weblogic, Spring, Maven, QCon, Guava, JPA, Colt, Zing, Pico, JAXB, Applets, Struts, Groovy, JavaFX, Log4J, BluRay, Glassfish, Tapestry, JavaRanch, JRoller, Fusion, Excelsior JET, JAX-WS, BIRT, JDOM, Yourkit, SmartCard, JTA, Fantom, Gradle, Netbeans, OSGi, CXF, JSTL, ActiveMQ, JEDI, Camel, JRuby, ServiceMix, Jython, Joda-Time, and many, many more!

It is a huge ecosystem - with a gigantic community

The investment by Sun, now Oracle, is miniscule compared to that of the rest of the community. Essential and vital, but miniscule.

And that is The Deal! The investment by Sun/Oracle (which is increasing under Oracle) is a whole lot less valuable if the community and ecosystem is destroyed. And that is why the recent Java Community Process debates really matter.

In order to function, there needs to be a safe base for everyone to work off. This has for many years been investment from Sun/Oracle moderated via the Java Community Process. But that deal has hit a major roadblock in the last few months.

The crazy part is that by damaging the community, Oracle's actions damage themselves! As The Deal indicates, Oracle's current increased investment, and the pushing forward of Java 7 and 8, is of limited value if the community and ecosystem ebbs away and ceases to make the platform relevant.

The lawsuits and JCP actions simply serve to push that community away, creating distrust. Some in the Oracle management chain need to think again about what community and ecosystem really means.

Sometimes it means taking decisions that are against the immediate and obvious business benefit in order to keep the community and ecosystem happy and productive, gaining a bigger business benefit later.

The message is simple. Everyone in the community and ecosystem, from the biggest company to the smallest individual matters. Even those who might be classified as "enemies" or "competition" are in fact business partners who are vital to Oracle's investment in Java.

That's what a symbiotic relationship means. And what The Deal summarises.

Stephen Colebourne
Personal opinion about the state of the Java community

Friday December 10, 2010

Is the JCP dead?

Today, the Apache Software Foundation (ASF) joined Doug Lea and Tim Peierls in resigning from the Java Community Process (JCP) Executive Committee (EC). The message from the ASF was simple - The JCP is Dead. But is it?

Is the JCP dead?

The JCP, is the Java Community Process, setup in 1998. But why did it get created at all?

Well, back in 1996 - 14 years ago - Sun started on the road of creating an globally recognised Open Standard for Java at the International Standards Organisation (ISO). This standard was a promise to the community, in part to gain adoption of the platform.

If you weren't involved in Java at the time this might seem unimportant. Yet I can remember using the promise of an Open Standard as one of the arguments for adopting Java as the principal language in the insurer I worked at. I'm sure other developer's similarly found the Open Standard argument useful in adoption discussions. That promise was responsible in part for the rapid adoption of Java.

Sadly, the tale of standardisation through ISO, and then ECMA, is a sordid one. This was a time when Microsoft feared and abused Java resulting in a lawsuit. Sun faced two main hurdles. Firstly, that it wouldn't give up its copyright, patents, trademarks or rights to the compatibility testing suite (which ISO wanted them to). And secondly, the very rational fear that its competitors, notably Microsoft and HP, would abuse the standards process to gain control of, and destroy, Java. (For full details, see this excellent academic paper).

The JCP was setup effectively after the efforts at ISO and before the efforts at ECMA. But initially, it was very focussed on vendors, with a minimum $2000 price tag per year. Some early criticism is still interesting:

To me this makes it pretty clear, that this process is not an open one. If you're still not convinced, ask yourself these questions:

0. Can anyone tell Sun No? Can anyone keep Sun from putting something into the spec they want to put it in? Or put something in that Sun wants to keep out?

1. Can Sun's enemies (e.g. Microsoft, HP, etc.) participate in this process on an equal footing with Sun? Can they even participate at all?

When you actually read the fine print, what the Java Community Process says is that Sun agrees to let other companies contribute their time, money, and knowledge to help Sun do what it wants to do anyway. That may be intelligent business, but it's not an open, community based process for developing standards.

In hindsight, Sun's choices to avoid ISO and ECMA look reasonable. Java was not that established, and fast moving technology often doesn't fit well in standards bodies. And the threat from Microsoft was very real. Thus the JCP was better than nothing. It wasn't an open standards body though.

The Apache Software Foundation (ASF) did not participate initially but later played a part. In the lead up to JavaOne 2002, the Apache Software Foundation worked with Sun to ensure that specifications from the JCP could be implemented in Open Source. On stage announcement (large mp4 file) and Press reports:

The biggest news of this year's JavaOne was delivered Tuesday morning by Jason Hunter, an Apache Software Foundation vice president, co-creator of JDOM, and author of the popular O'Reilly Servlets book. Flanked by Sun CEO Scott McNeally and Sun vice president Rob Gingell, Hunter outlined an agreement negotiated between Sun and Apache that has broad-ranging implications to developers and to the future of Java itself. First, all in-progress and future Sun-led Java Specification Requests (JSRs) will be made available under a license that allows for open source.

In addition, key JSRs that have already been released will have their license altered to this newer, more open license. Even JSRs not being led by Sun can release their reference implementations under an open source license. Finally, Test Compatibility Kits (TCKs) for Sun-led JSRs will be made available for free to qualifying open source and academic groups.

There is a further important implication of Sun's statement that "Sun will modify the specification licenses of all the JSRs currently in progress to reflect Apache's requirements as met in the new draft JSPA." The JSPA is the legal agreement signed by members when they join the Java Community Process. Hunter points out that the draft JSPA includes the requirement that the TCK be licensed separately from the Reference Implementation (RI). In the past, commercial interests, which used to have to license the RI and TCK together, were bound by some interesting legal issues by having the RI source. Under the new agreement, they can now pay just for the TCK. This will help JBoss, for example, which can't get the TCK because it's currently only provided with the RI; they can't see the RI and remain independent.

This agreement from 2002, and the associated updated JSPA legal agreement, are the defining legal basis for the ASF's actions today.

The actual JSPA says.

Other than as set forth above, the Spec Lead agrees not to impose any contractual condition or covenant that would limit or restrict the right of any licensee to create or distribute such Independent Implementations.

Since 2002, the ASF, and many others, have implemented JSRs using these rights. This includes Apache Tomcat, Geronimo, OpenEJB, MyFaces and many more. The ecosystem worked! And surely the community now had an Open Standards body for Java equivalent to ISO?

But, when the ASF started the Harmony project, Sun was not happy.

It decided to stop Harmony using a Field Of Use restriction on the testing compatibility kit. The aim was simple, if Harmony passed the tests, then it wouldn't be open source software. Thus the ASF couldn't release Harmony. It was legally clever. Except that it conflicted with the JSPA legal agreement, as outlined above.

After private negotiations went nowhere, the ASF talked to the JCP Executive Committee (EC). The EC supported the position of the ASF in two critical votes. Oracle voted in support of the position of the ASF in both votes.

By this stage, Sun was in financial trouble. The apparent threat from Harmony was that the Java ME licensing revenue would disappear, and since every last dollar mattered, Sun did not feel able to concede. Instead, Sun released OpenJDK as the GPLv2 implementation of Java, placating some like Richard Stallman.

But the dispute itself dragged on. And on. And on.

Until Oracle bought Sun.

At that point there was hope that the issue would be resolved. After all, Oracle's public position was in support of the ASF. But over time it became apparent that Oracle intended to make no change to the licensing offered. It had done a U-turn.

So why didn't the ASF sue Sun, then Oracle? Well, basically its a really bad idea for a not-for-profit to sue a mega corporation, no matter how good the case. (That said, it might still have been the better course of action)

And why did the Executive Committee fail to fix the problem? Because there is no dispute resolution process in the JCP. All legal agreements are between the member and Sun, now Oracle - because the JCP itself is just a department, not an actual legally recognised organisation.

What the EC could do was prevent the approval of the Java SE 7 JSR. And they did. While this isn't the whole reason why its been so long since Java SE 6, it is a major part of the story.

Once Oracle finally got to grips with the issue it made a decision to force Java SE 7 and 8 through, no matter what. Tim Peirels captured this in his resignation statement:

Add to that Oracle's expressed intent to proceed with the SE7/8 JSRs whatever the outcome of the vote, and one can only conclude that the SE/EE EC is never going to be more than a rubber stamp for Oracle. (The belligerent tone with which this message was delivered does not come across in the public minutes, but it was loud and clear over my phone connection.)

At the same time, the second strand of Oracle's plan was in action. IBM and Apple were being invited to join OpenJDK in leadership positions. Effectively, this was the sign that IBM would now vote through the Java SE 7 and 8 JSRs, irrespective of the ASF dispute.

However, even I was surprised at just how clear the mismatch between the Java SE 7 license and the JSPA was. Not only that, the license terms are inconsistent with the stated uses of the JSR itself.

Of course, none of this mattered. The EC corporate members, and their lackie Eclipse, had decided to pass the JSRs even though they were in violation of the legal agreements. This was now big company politics, and if the ASF had to go, then it had to go.

Doug Lea resigned just before the vote. Tim Peirels just after. And today, the ASF joined them.

After all, there is no point helping to write specifications that you aren't allowed to implement.

But were they right to declare the JCP is dead?

The first ASF claim is that the JCP is not an open standards organisation because there are restrictions made in the commercial interests of one party that prevent another from producing an implementation. The second ASF claim is that the EC failed to perform its duty to enforce the rules and to defend a fellow member.

By approving Java SE 7, the EC has failed on both counts : the members of the EC refused to stand up for the rights of implementers, and by accepting Oracle's TCK license terms for Java SE 7, they let the integrity of the JCP's licensing structure be broken.

The Apache Software Foundation concludes that that JCP is not an open specification process - that Java specifications are proprietary technology that must be licensed directly from the spec lead under whatever terms the spec lead chooses; that the commercial concerns of a single entity, Oracle, will continue to seriously interfere with and bias the transparent governance of the ecosystem; that it is impossible to distribute independent implementations of JSRs under open source licenses such that users are protected from IP litigation by expert group members or the spec lead; and finally, the EC is unwilling or unable to assert the basic power of their role in the JCP governance process.

But none of this definitively means that the JCP is dead.

Oracle's public argument is that Java needs to move forward:

Last month Oracle renominated Apache to the Java Executive Committee because we valued their active participation and perspective on Java. Earlier this week, by an overwhelming majority, the Java Executive Committee voted to move Java forward by formally initiating work on both Java SE 7 and SE 8 based on their technical merits. Apache voted against initiating technical committee work on both SE 7 and SE 8, effectively voting against moving Java forward. Now, despite supporting the technical direction, Apache have announced that they are quitting the Executive Committee. Oracle has a responsibility to move Java forward and to maintain the uniformity of the Java standard for the millions of Java developers and the majority of Executive Committee members agree. We encourage Apache to reconsider its position and remain a part of the process to move Java forward. ASF and many open source projects within it are an important part of the overall Java ecosystem.

For me, this is very hollow. The reference to "technical merits" covers the fact that many members used their comments to object to the licensing. And frankly, Oracle were going to proceed whether or not the EC voted "Yes".

My conclusion is that the EC should have voted "No" and forced Oracle to run Java SE 7 and 8 outside the JCP, thus preserving the integrity of the EC and the JSPA license. And this action would have had zero effect on "moving Java forward".

Since the EC voted "Yes", they tacitly accepted Oracle's right to break the JSPA legal agreement when it is in their commercial interest.

Given that the ASF has no commercial axe to grind, they made the sensible choice to leave. And since the EC failed to act as the leaders of the industry it is entirely reasonable to say that the JCP is now a sham body where legal agreements mean little.

Thus essentially whether you consider the JCP is dead or not depends on what you believe the JCP is.

Since 2002, I have essentially seen the JCP as a vendor-neutral open standards body, the Java specific equal of ISO. Clearly, it can no longer be viewed as that.

But others have viewed it more as a useful tool for influencing the owner of Java, where the output sometimes allows the creation of a competitive ecosystem around a specification. Viewed this way, the JCP is still entirely feasible.

Certainly, Doug Lea saw it as the former:

I believe that the JCP is no longer a credible specification and standards body

So, is the JCP dead? Well, it will probably limp on for a while. And if the second view of what it is takes hold, then it may have some future as far as big corporates are concerned.

But it cannot now be seen as an Open Standards Body.

So its both dead and undead. A zombie.

Summary

Its the end of an era. An era of hope that the JCP would be Java's ISO, producing truly Open Standards.

So as that era closes we look forward to the new closed era of Java. Where its Oracle's way or the highway.

Stephen Colebourne
Apache Software Foundation member, speaking personally
Oracle Java Champion, speaking personally
Not a committer on Harmony or OpenJDK

Tuesday December 07, 2010

Java SE 7/8 passed

So, the Java SE 7 and 8 votes passed.

Java SE 7 passed by 12 votes to 3 (Apache, Google and Tim Peierls voting against).

Java SE 8 passed by 12 votes to 3 (Apache, Google and Tim Peierls voting against).

Project Coin passed by 13 votes to 1 with 1 abstention (Apache voting against and Tim Peierls abstaining).

Project Lambda passed by 13 votes to 1 with 1 abstention (Apache voting against and Tim Peierls abstaining).

But it looks like many of the voters were unhappy.

Java SE 7 vote results

When voting for a JSR there is the opportunity to leave comments. These often indicate the true feelings of voters more than the actual Yes or No. Here are the actual vote comments for Java SE 7 (the Java SE 8 comments are similar but with reference to OSGi):

On 2010-12-03 Peierls, Tim voted No with the following comment:
Changing my vote from "Abstain" to "No" to register my extreme disappointment with Oracle's failure to address EC questions about the licensing terms of this JSR.
------------------------------------------------------------------------------
On 2010-11-17 Google Inc. voted No with the following comment:
While we support the technical content of this JSR, Google is voting no because of its licensing terms. As per the JCP resolutions of 9/25/2007 and 4/7/2009, "TCK licenses must not be used to discriminate against or restrict compatible implementations of Java specifications by including field-of-use restrictions on the tested implementations or otherwise. Licenses containing such limitations do not meet the requirements of the JSPA, the agreement under which the JCP operates, and violate the expectations of the Java community that JCP specs can be openly implemented." Most EC members, including Oracle, voted for both of these resolutions. But the proposed license clearly violates them in Exhibit A, Section II.

It would be wrong to condone the inclusion of field-of-use restrictions in a TCK license (which violates the above resolution and the JSPA) by voting for this JSR. We were initially reluctant to vote no because we do not want to delay progress of the Java platform. But this concern was made moot by Oracle's statement at the JCP meeting of 10/4/2010 that they intend to move forward with the release outlined in this JSR with or without the approval of the JCP.
------------------------------------------------------------------------------
On 2010-11-17 Oracle voted Yes with no comment.
------------------------------------------------------------------------------
On 2010-11-29 SAP AG voted Yes with the following comment:
While we are disappointed that Oracle has decided to deny Apache a TCK license for Java 7, SAP's vote is strictly based on the technical merits of the Java 7 specification, not on its license terms. While we believe it is important for Java 7 to proceed now, we want to express our disagreement about Oracle's decision regarding the TCK for Apache. The reason that was provided to the EC was that Java compatibility could no longer be enforced with code shared under a permissive open source license. As the new steward of the Java language, we respect Oracle's right to license their intellectual property under terms that support their strategy of how to evolve the Java ecosystem. However, we believe that the additional potential for innovation in open source communities outside of OpenJDK well offsets the risks as already demonstrated in the Java Enterprise space.
------------------------------------------------------------------------------
On 2010-11-30 Hewlett-Packard voted Yes with no comment.
------------------------------------------------------------------------------
On 2010-12-02 IBM voted Yes with the following comment:
IBM's vote is based on the technical merits of this JSR and is not a vote on the licensing terms. IBM supports licensing models that create an open and level playing field by allowing third parties to create independent implementations of Java Specifications and that do not allow individuals or companies to exercise unnecessary control for proprietary advantage. We support open source as a licensing model for contributions in the JCP, and would hope others will support this direction. This comment is not necessarily directed at the current business or license terms for this JSR, however, it is a statement of IBM's preferred licensing model.
------------------------------------------------------------------------------
On 2010-11-30 Ericsson AB voted Yes with no comment.
------------------------------------------------------------------------------
On 2010-12-01 Apache Software Foundation voted No with the following comment:
The Apache Software Foundation must vote no on this JSR. While we support the technical contents of the JSR, and earnestly support the need for the Java platform to move forward, we cannot in good conscience vote for this JSR because :

a) This JSR's TCK license includes a "Field of Use" restriction that restricts commonplace and mundane use of independent implementations, a licensing element that not only is prohibited by the JSPA but also has been unanimously rejected by the majority of the members of the JCP EC - including Oracle - on several occasions. We can only speculate why Oracle included such an element, but we believe that an open specification ecosystem must be independent of - and protected from - any entity's commercial interests.

b) On process grounds, this JSR is in conflict with it's own TCK license. The JSR explicitly states that Java SE is targeted for, among others, embedded deployments. Yet the TCK license specifically prohibits such usages (for example, netbooks) of tested independent implementations. We find this to be a misleading legal trap for potential implementers, and believe that any independent implementation that passes the TCK should be able to be used and distributed under whatever terms deemed fit by the implementer.

c) The spec lead has ignored repeated requests from multiple EC members for and explanation of both a) and b)

d) The spec lead - Oracle - is in breach of their obligations under the JSPA by continuing to provide a TCK license for Apache Harmony under terms that allow Apache to distribute its independent implementation under terms of its choice. We do not believe that anyone that willfully fails to meet their contractual obligations under the JSPA should be allowed to participate as a member in good stating in the JCP. The rules apply to everyone.

While we understand that it's Oracle's stated intent to move forward irrespective of the EC's decision, we urge Oracle to fix the above-mentioned issues, and continue to work with the members of the JCP within the structure of the JCP to keep Java a vital and viable platform.
------------------------------------------------------------------------------
On 2010-12-02 Eclipse Foundation, Inc voted Yes with the following comment:
Eclipse is disappointed with the continuing issues around Java licensing. The unresolved TCK licensing issue with the Apache Software Foundation is a symptom of the fundamental problem with FOU restrictions on the Java platform. Our vote is based entirely on the premise that improvements and evolution are required if the Java platform is to remain viable. In addition, we remain concerned that the future of Java modularity may fail to adequately reflect the requirements of the existing OSGi community and ecosystem. Future votes on these JSRs will reflect this concern.
------------------------------------------------------------------------------
On 2010-12-02 Fujitsu Limited voted Yes with no comment.
------------------------------------------------------------------------------
On 2010-12-04 RedHat voted Yes with the following comment:
Red Hat's vote is based solely on the technical merits of the JSR. We believe that this JSR is important for the industry at large and that further delays risk fracturing the Java landscape further. However, we are extremely disappointed with the license terms and that a more open license has not been adopted by the Specification Lead. We continue to believe that the TCK should contain no "field of use restrictions", as originally raised by Apache with regard to another JSR during the development of EE6 (i.e. the SE TCK licensing). As a result, this approach risks limiting adoption of this and subsequent JSRs by a wider audience and the fracturing that we mentioned earlier could still occur. We hope that all Specification Leads will take this into account in the future and rank the viability of the Java Community higher than one individual member's abilities to monetise the platform.
------------------------------------------------------------------------------
On 2010-12-05 VMWare voted Yes with no comment.
------------------------------------------------------------------------------
On 2010-12-06 Keil, Werner voted Yes with the following comment:
While I'm equally dissapointed with the ongoing license dispute and the effect this and other issues had on the actual content and scope of Java 7 (I am not the only one who feels this is technically hardly worthy of a version major compared to especially Java 5) I believe, it is a necessary and long overdue step towards the future of Java. I share discomfort with less talked about, but nevertheless tedious problems like the lack of modularity in SE7 and its effect on dependent JSRs like EE.
------------------------------------------------------------------------------
On 2010-12-06 Intel Corp. voted Yes with no comment.
------------------------------------------------------------------------------
On 2010-12-06 Credit Suisse voted Yes with the following comment:
Credit Suisse's vote is purely on the technical content. We strongly demand open standards and an active community around Java as we selected Java SE & EE as primary pillars for our application development (as many others in the industry do). The current battle around licensing term, however, reveals that Java never actually was an open standard. FOU restrictions clearly discriminate open source implementations and prevent competition, and with that, innovation in that space. While Java had a considerable head start, it lost a lot of momentum over the last years. Fragmentation (or a fork) of the language and its platforms are clearly not desired. But today, customers are already facing competing models for developing enterprise applications (e.g., Spring, OSGi, Java EE). The main problem, in our view, is the lack of modularization, clear delineation of Java IPs owned by Oracle and truly open standard extensions, and the ignorance of developments outside of the JCP (even though OSGi has a JCP blessing). The OpenJDK framework is not sufficient for all aspects of the language. Java must be kept interesting for researchers and universities: researchers not only contribute to the standards (e.g., Doug Lea for concurrency or Michael Ernst for type annotations) but also decide on the languages (and paradigms) that are taught at universities -- and this in the end determines the knowledge and mindset we acquire with our software engineers. While we recognize Oracle’s intellectual properties around Java, we strongly encourage Oracle to re-think its current position around licensing terms. We strongly support open source as a licensing model for contributions in the JCP.

------------------------------------------------------------------------------

There is a lot in those comments to understand, and I can't cover it all now.

I do note Time Peierls decision (as an individual) to change his vote from "Abstain" to "No" and the reason given. Clearly, Oracle were asked to explain their licensing terms and how they square with the JSPA, and with the JSR text itself, and have failed to do so. My interpretation is that Oracle no longer felt the need to justify its actions.

I also castigate Hewlett-Packard, Ericsson AB, Fujitsu, VMWare (SpringSource) and Intel for not even having the courage to justify their actions (which are in direct opposition to previous votes). I will note that Intel has previously been a strong Apache supporter. However, I understand that the Intel representative on the JCP was changed a few months back, prior to the vote (no public source to confirm this as far as I know Update: the executive committee members list has been updated to show the change, which happened since July).

I am disappointed in all the "Yes" voters. Clearly Oracle intended to move Java SE 7 forward irrespective of the vote here. Thus voting "No" would have had no adverse consequences on the forward momentum and technical progress of Java. Since the main role of the Executive Committee, distinct from Expert Groups, is to consider the business implications it is clear that each company that voted "Yes" has given tacit approval to the business practices of Oracle. I suspect that they will come to regret that decision. Give a bully an inch and they will take a mile.

Summary

Today, Oracle won its battle in the JCP. But it was a stupid battle to have. Submitting a JSR whose licensing terms are in conflict with the JSR itself and with the JSPA was simply unnecessary. Voting "Yes" to it was simply cowardly.

The huge irony is that Oracle claims to support Java because all of its money making systems are written in Java, thus it is in its own interests to keep Java vibrant. Yet the strong-arm actions here simply push people away and harm that vibrancy. Go figure.

We should expect the response from the Apache Software Foundation soon.

Stephen Colebourne
Apache Software Foundation member, speaking personally
Oracle Java Champion, speaking personally
Not a committer on Harmony or OpenJDK

Saturday November 20, 2010

Devoxx whiteboard votes 2010

As usual at Devoxx, there were a number of Whiteboard polls. These aim to capture the sense of the community and are of course only semi-scientific!

This year, picking questions was a little tricky, given the current disputes. I decided to ask questions focussed on specific topics and provide a choice of responses. Of course, you can never cover all valid responses, so there was always a comment area for extra opinions.

The +n notation after a comment means that n other people agreed with the comment. The <- notation means that what follows is a comment on the previous comment.

How do you feel about the Oracle vs Google lawsuit?

"It does not affect me. I do not care"	10
"It is necessary to avoid fragmenting the Java platform"	16
"There may be a case but legal action is a bad idea"	62
"I really care - its a dumb idea"	116
"It is encouraging me to look at non-Java platforms"	9
Total votes cast	213

View the photo. There were also some comments:

• The lawyers will win. +3
• Google should buy Java/Oracle
• Google should create/drop Java+
• $$Larry Ellison needs a new yacht. +2
• Ban software patents. +7
• Dimemma: Its fun to watch
• Big powerplay for mobile space
• Crazy SCO move!
• Fight!
• I know a good mediator: Equilon
• Patent trolls
• Patent should help small reality to contribute! Not help huge industry to black mail each other! <- (Ponies and Rainbows anyone?)
• What's JSR nnumber for Android!

A very clear answer was provided on this topic - devoxx attendees that voted were solidly against the lawsuit (over 87%). While a high number thought there might be a case even more thought the lawsuit was just plain dumb. If Oracle wants developers on its side, it is clear that the lawsuit is a problem. As always, the comments were entertaining too.

Should Java 8 or 9 be backwards incompatible?

"Yes! Its time to remove some old stuff to make room for new"	185
"Maybe! Some SMALL incompatible changes would be good"	35
"No! Backwards incompatibility is essential to Java"	27
"I don't care, I'll just use whatever I am given"	4
Total votes cast	251

View the photo. There were also some comments:

• Java NG, Bytecode should still be usable. +5
• Please cleanup IO
• Cleanup. Pe Java 5 stuff
• Remove java.util.Date. +1
• Corbe pollutes APIDOC <- so use Jigsaw
• Remove ESB2 compatibility +1

Clearly, a majority of voters wanted some form of backwards incompatibility (over 87%). This is a remarkably clear result and was more positive than I expected.

How do you feel about the JCP?

"It is very important and will continue to be"	3
"Everyone needs to start working together"	36
"It needs radical reform"	24
"It is no longer relevant"	5
"It is F***ed" (added by a conference attendee)	12
"I don't know anything about it" (added by a conference attendee)	8
Total votes cast	88

View the photo. There were also some comments:

• Who cares? OSS already demonstrates "de facto" standard to override JSR. +2
• Hibernate is OS,Hibernate gave birth to JPA, JPA is standard, Hibernate implements the standard (<- Mostly), Hibernate is still OS
• Design by committee
• Will be James Brown, doubt it. "Get in it, Get involved, Everybody get funky"
• Only good for big vendors, at the detriment of developers
• Get out of the way. Turn in to a real open COMMUNITY!
• The "new" mantra is cool: "Test stuff in the community first then standardize what really works"

Opinion was split here, on a relatively low turnout. Clearly compromise is still valued, but some believe it may be too late.

Does Oracle believe in the Java community?

"Yes! They are investing $millions"	10
"Maybe! They just see it differently to Sun"	83
"No! They are severly damaging the community"	36
"I don't care, the community is not important"	2
Total votes cast	131

View the photo. There were also some comments:

• Bad communication, but they really wat to learn. +1
• Google and ASF should join forces and start working on BIJava cause Oracle will never have the balls to do it
• JavaOne needs to be 1st class citizen again! +1
• Demonstrate first: Free talk +3
• Trust must be earned
• Larry Ellison pick up the phone, Eric Schmidt pick up the phone, Geir Magnusson Jr pick up the phone, DEAL
• <-(someone forgot to call me!)
• Java community is watcing! If Oracle f****up something new might come out and takeover
• Communication is the key!

The message here seems to be that many are still willing to give Oracle a chance. I'm interested in those that do not believe community is important ;-)

How should Oracle make money from Java?

	Paid for	$Free
"Core JVM"	0	39
"JVM tools (JRockit mission control)"	26	10
"IDEs"	7	24
"Application server (Webloic)"	30	4
"Middleware (ESB/Fusion)"	31	3

View the photo. There were also some comments:

• Not +2 ("not" as in should not make any money I presume)
• Stop paying lawyers! <- LOL
• In whatever way works to max income

This vote only started on Thursday, hence the lower voting figures. The results ought to be predictable, but clearly a few want $free everything. In general it looks like Oracle's paid vs $free choices are about right.

OpenJDK is now the "heart" of Java

"I will contribute to OpenJDK"	12
"I like OpenJDK but don't want to contribute"	16
"The OpenJDK GPL license is uncertain because of Oracle's patents"	31
"I don't believe OpenJDK should be the only Java SE implementation"	37
Total votes cast	96

View the photo. There were also some comments:

• Troll? +1
• A clear statement from Oracle is needed on WHY they do not want to make it possible for others to have Java implementations (Harmony). Would like to hear BUSINESS reasons. +8
• <-Maybe mreinhold could explain?
• More communication is what will solve the problems & stop damaging speculation
• Oracle should make TCK available w/o field-of-use restrictions. +5
• Harmony FTW, keeps them true. +1
• It should be. +1
• Should be LGPL

There is definitely support here for other Java SE implementations, yet not a huge number of votes overall. I suspect that amongst those that know the story it matters, but in general it doesn't. I also find the (popular) comment for an explanation interesting.

What is your favourite non-Java language?

Scala	69
Groovy	42
Python	22
Javascript	22
Ruby	16
Clojure	8
Objective C	7
Smalltalk	7
PHP	5
C	5
C++	5
Perl	5
Fantom	3
C#	3
Forth	3
Z80 assembler	3
Human language	3
Body language	2
CSS	2
Pascal	2
BAS#	2
Haskell	1
Visage	1
Erlang	1
Modula 3	1
MPS	1
Prolog	1
Eiffel	1
InterCal	1
MatLab	1
BPMN 2	1
Lisp	1
LiveCode	1
HyperTalk	1
Total votes cast	244

View the photo. Last year Groovy came top, this year it is Scala. One point to bear in mind is that Groovy conferences are now well established, so possibly those most dedicated to Groovy nonw attend those. Or maybe Scala is getting more popular. And I was personally surprised to see how popular Python is.

Summary

Devoxx votes are nothing more than a snapshot of opinion by a proportion of the attendees of Devoxx. Yet despite the caveats, their results can be interesting. Its the power of the community!

Wednesday November 17, 2010

Java SE 7 and 8 JSRs published

Following yesterday's statements from Oracle and Apache, the meaty JSRs are now published and announced by Mark Reinhold.

I’m pleased to note the submission of four new Java Specification Requests to the Java Community Process:

* JSR 334: Small Enhancements to the Java Programming Language, by Joe Darcy with help from Jon Gibbons, Maurizio Cimadamore, and many others in Project Coin;
* JSR 335: Lambda Expressions for the Java Programming Language, by Brian Goetz with help from Alex Buckley, Maurizio, and others in Project Lambda;
* JSR 336: Java SE 7 Release Contents, for the enormous team effort that is JDK 7 (the first half of Plan B); and, finally,
* JSR 337: Java SE 8 Release Contents, for the eventual JDK 8 (the rest of Plan B).

These JSRs have been a long time coming. They’re now—finally—on the JCP Executive Committee ballot for approval; results should be available in two weeks.

My thanks to everyone who’s contributed to the related OpenJDK Projects.

Cool! Java is moving forward! And JSR-310 is proposed for Java SE 8!

But, lets take a peek at the interesting stuff... the licensing terms. (OK, its not really that interesting unless you want to understand the raging Java disputes...)

Firstly, the licenses are provided in Word Document format! (See section 2.18 of the JSR page for the links)

Its late here, so I only have limited time to evaluate the licenses. At first glance, the Specification and Reference Implementation licenses look fairly normal and bland (but I may be wrong). The interesting one is the TCK (testing kit) license.

I emphasise that this should be considered to be notes on the TCK license based on a relatively quick reading. If you read this blog and reference this topic/story in another blog or article, you are advised to read and examine the original license text yourself, or include a suitable caveat.

The famous "Field of Use" is an explicit part of the license:

1.4 "Exhibit A" means collectively Exhibits A-1 through A-n which incorporate into the Agreement the specific terms and conditions for each TCK licensed hereunder.

1.6 "Field of Use" means the relevant market segments for products tested by a particular TCK for a Java Environment Specification as specified in the applicable Exhibit A(s).

The "Java Environment Specification" is defined, covering Java SE:

1.8 "Java Environment(s) Specification" means a Java Specification that defines a baseline API set that provides a foundation upon which applications and other Java Specifications can be built. For example, and not by way of limitation, Java Environment Specifications include: (a) “Platform Editions” such as the Java Platform, Standard Edition ("Java SE"); Java Platform, Enterprise Edition (“Java EE”); and Java Platform, Micro Edition (“Java ME”) Specifications; (b) “Configurations” such as the Connected Device (“CDC”) and Connected Limited Device (“CLDC”) Configurations; and (c) “Profiles” such as the Mobile Information Device (“MIDP”) Profile.

The definition of a "product" contains what looks like an unusual part (highlighted). It appears that a "product" must meet three criteria beyond the basic ones:

• "have a principal purpose which is substantially different from a stand-alone implementation of that specification"
• "represent a significant functional and value enhancement over any stand-alone implementation of that specification"
• "not be marketed as a technology which replaces or substitutes for a stand-alone implementation of that specification"

I believe that Apache Harmony would fail all three of these tests (were the project to try and implement this JSR, which they probably won't). Since a "stand-alone implementation" would be either OpenJDK or OracleJDK, the principal purpose of Harmony is clearly the same (not substantially different), Harmony does not offer significant functional enhancement, and Harmony would be marketed as a replacement for OpenJDK/OracleJDK.

1.12 "Product(s)" means a Licensee product which: (i) fully implements the Java Specification(s) identified in Exhibit A including all its required interfaces and functionality; (ii) does not modify, subset, superset or otherwise extend the Licensor Name Space, or include any public or protected packages, classes, Java interfaces, fields, methods or constructors within the Licensor Name Space other than those required/authorized by the Specification or Specifications being implemented; (iii) passes the TCK (including satisfying the requirements of the applicable TCK Users Guide) for such Specification; and (iv) neither derives from nor includes any of Oracle’s source code or binary code materials which implement any portion of the Java Specification, except for code contributed by Oracle to an open source project (e.g. Apache’s “Tomcat” project) and that is rightfully (i.e. pursuant to a separate and appropriate license) included in the product to be tested. In addition, to be a Product, a Licensee product that implements a Java Environment Specification must: (a) have a principal purpose which is substantially different from a stand-alone implementation of that specification, while the value-added portion of the product operates in conjunction with the portion that implements the Java Environment Specification; (b) represent a significant functional and value enhancement over any stand-alone implementation of that specification; and (c) not be marketed as a technology which replaces or substitutes for a stand-alone implementation of that specification.
(My highlights)

Criteria are given as to what not to do. This indicates that no one may distribute code that implements the specification unless it is part of a product, as defined above. Given Apache Harmony cannot be a product, it cannot be distributed.

(b) Additional Limitations. Except as otherwise set forth in Exhibit A, Licensee may not:
...
(v) distribute code which implements any portion of the Java Specification unless such code is included in a Product within the meaning of Section 1.12 and unless, for each new release of a Product by Licensee, such Product passes, in accordance with the Documentation (including the TCK Users Guide), the most current TCK applicable to the latest version of the Java Specification and available from Oracle one hundred twenty (120) days before FCS of such version of the Product; provided, however, that if Licensee elects to use a version of the TCK also provided by Oracle that is newer than that which is required under this Section 2.1(b)(v), then Licensee agrees to pass such TCK;
(My higlights)

Were a court case to challenge the contents of the license, Oracle can at its discretion terminate the licence.

11.4 Partial Invalidity. If any of the above provisions are held to be in violation of applicable law, void, or unenforceable in any jurisdiction, then such provisions are herewith waived or amended to the extent necessary for the Agreement to be otherwise enforceable in such jurisdiction. However, if in Oracle's opinion deletion or amendment of any provisions of the Agreement by operation of this paragraph unreasonably compromises the rights or increase the liabilities of Oracle or its licensors, Oracle reserves the right to terminate the Agreement.

And Exhibit A contains the actual "Field of Use" terms. These specifically exclude wireless mobile telephones, kiosks and even netbooks (highlighted). Remember that any Field of Use restriction violates the OSI open source definition.

EXHIBIT A
TECHNOLOGY SPECIFIC TERMS AND CONDITIONS

I. Description of TCK, Test Tools and Documentation

A. Java Specification: Java Platform, Standard Edition, version 7 (“Java SE 7”), which includes mandatory standalone elements to the extent described and permitted in the Java SE 7 specification and which includes optional elements to the extent described and permitted in the Java SE 7 specification
...
II. Field(s) of Use: Products for use on “General Purpose Desktop Computers and Servers" meaning computers, including desktop and laptop computers, or servers, used for general computing functions under end user control (such as but not specifically limited to email, general purpose Internet browsing, and office suite productivity tools). The use of Software in systems and solutions that provide dedicated functionality (other than as mentioned above) or designed for use in embedded or function-specific software applications, for example, but not limited to: Software embedded in or bundled with industrial control systems, wireless mobile telephones, wireless handheld devices, netbooks, kiosks, TV/STB, Blu-ray Disc devices, telematics and network control switching equipment, printers and storage management systems, and other related systems are excluded from this definition.
(My highlights)

The published release must contain a note from Oracle emphasising the conditions on further distribution:

1. The following provision is added as subparagraph (viii) to the Additional Limitations set forth in Section 2.1(b):

(viii) distribute Products unless accompanied by the following notice from Oracle, where the notice is displayed in a manner that anyone receiving the Product will see the notice:

NOTICE FROM ORACLE:
If you redistribute the software licensed hereunder (including derivative works thereof) for your direct or indirect commercial gain, then we are not authorized to grant or otherwise pass through to you any licenses under Oracle applicable intellectual property or other rights, if any, and as a result any such use is a violation of Oracle’s applicable rights. Any redistribution of the software licensed hereunder (including derivative works thereof) must be compatible and branded with the appropriate compliance logo specified by Oracle and licensed by Oracle to you pursuant to a separate Trademark License required to be executed by you with Oracle.

Redistribution of the software licensed hereunder must retain this notice.

The cost of the TCK is specified. And it is cheap. So, none of the dispute is directly about money (indirectly is another matter!).

A. For Commercial Licensees: $100,000.00 (as of the Effective Date) per year. All fees shall be due upon execution of this Agreement. In addition, Oracle shall have no obligation to deliver or make available the TCK until such fees are received by Oracle.

B. For Qualified Not-for-Profits and Qualified Individuals: $0.

And thats it. Well, at least the parts I see as being interesting. But if you want to comment on this, you really should read it yourself!

Sadly, at 3am, I don't have time to contrast the TCK license with the JSPA.

To be honest, I'm surprised that the TCK license for Java SE 7 still contains any pretence that it can be implemented in open source by anyone other than Oracle. At least the restrictions are clear (and I suspect, but cannot prove, that very similar restrictions were offered for Java SE 5 in the Sun/Oracle vs Harmony dispute).

I personally expect these 4 JSRs to pass the JCP Executive Committee vote in two weeks time as I see no evidence that committee members are still up for a fight. If the vote passes, I expect the ASF will immediately leave the JCP.

I will leave the biggest amusing point until last. The TCK license conflicts with the JSR! The JSR itself says:

2.2 What is the target Java platform? (i.e., desktop, server, personal, embedded, card, etc.)

This JSR defines a release of the Java SE platform targeted at embedded, desktop, server, and cloud environments.
(My highlights)

So, the JSR is targetted at embedded environments! That's not what the TCK license says ;-)

Monday November 15, 2010

Oracle replies to the ASF

Oracle have responded to the Apache Software Foundation board statement.

Oracle nominated Apache for a seat on the Executive Committee in recognition of Apache's continued participation and valued contribution to the community. The recently released statement by the ASF Board with regard to their participation in the JCP calling for EC members to vote against SE7 is a call for continued delay and stagnation of the past several years. We would encourage Apache to reconsider their position and work together with Oracle and the community at large to collectively move Java forward. Oracle provides TCK licenses under fair, reasonable, and non-discriminatory terms consistent with its obligations under the JSPA. Oracle believes that with EC approval to initiate the SE7 and SE8 JSRs, the Java community can get on with the important work of driving forward Java SE and other standards in open, transparent, consensus-driven expert groups. This is the priority. Now is the time for positive action. Now is the time to move Java forward.
Don Deutsch, Vice President of Standards and Architecture

Update 16 Nov: And the ASF responded to the response!!! Here it is:

Oracle statement regarding Java: "Now is the time for positive action (and) to move Java forward."

The ball is in your court. Honor the agreement.

I've used this blog to highlight the role of politics in the development of Java, especially in this dispute. Personally, I'm glad that this story is coming to an end, as I can write more code and less blogs!

Basically, I see this statement from Oracle as expected and well written. Expected, because it simply states the known position. Well written, because it puts a spin on what Apache actually said.

The key sentence is "Oracle provides TCK licenses under fair, reasonable, and non-discriminatory terms consistent with its obligations under the JSPA."

Bear in mind that the TCK offered would result in the tested Harmony code not being able to be released under any open source license, not just the Apache license. See my previous detailed explanation with pictures. I'm almost amused that anyone would describe those terms as "fair, reasonable, and non-discriminatory".

Actually, the sentence doesn't talk about Harmony at all. Technically, it doesn't claim that any fair terms were offered to Harmony. All the sentence actually claims is that Oracle provides TCK licenses under fair terms, which is true for every other JSR expect Java SE. Yes, I really am reading this statement as a lawyer could!

I'd also point out Don's previous actions from 2007:

Resolution 1 (proposed by Oracle, seconded by BEA)
"It is the sense of the Executive Committee that the JCP become an open independent vendor-neutral Standards Organization where all members participate on a level playing field with the following characteristics:
* members fund development and management expenses
* a legal entity with by-laws, governing body, membership, etc.
* a new, simplified IPR Policy that permits the broadest number of implementations
* stringent compatibility requirements
* dedicated to promoting the Java programming model

Furthermore, the EC shall put a plan in place to make such transition as soon as practical with minimal disruption to the Java Community."

Oracle (Don) voted yes

Resolution 2 (proposed by BEA, seconded by Intel)
"The SEEE EC requests Sun to submit a Java SE7 JSR with the following characteristics:
* EG members will share costs by contributing to the TCK and RI submitted under common licensing terms that will allow any implementer to use them;
* accompanied by public spec, tck, ri licenses at JSR initiation;
* licenses do not contain Field of Use restrictions on spec implementations. * the JSR is operated by the Spec Lead under democratic principles (e.g. Java SE6)

Furthermore, from the time of submission, TCK license(s) for Java SE5 and later will be offered without field of use restrictions on spec implementations enabling the TCK to be used by organizations including Apache."

Oracle (Don) voted yes

Only Don can truly answer why his opinion changed once he was in a position to action what he called for.

So what next?

Well, barring an earthquake, the Java SE 7 vote will proceed, the vote will go Oracle's way, and the ASF will leave the JCP entirely. I do not see either the ASF or Oracle slowing down the Java SE 7 timetable at this point.

Finally, we sometimes need to remember that Oracle continues to invest heavily in Java, from ME to SE to EE to FX. We all want an end to stagnation. I simply object to the approach chosen to end the stagnation.

Stephen Colebourne
Apache Software Foundation member, speaking personally
Oracle Java Champion, speaking personally
Not a committer on Harmony or OpenJDK

Tuesday November 09, 2010

ASF ready to leave JCP

The Apache Software Foundation board has released a statement regarding the JCP.

The Apache Software Foundation (ASF) is proud to announce that it has been ratified for another three-year term on the Java Community Process (JCP) Executive Committee. Receiving support from 95% of the voters, this election allows the ASF to continue its 10 year effort to help bring transparency and openness to the JCP as well as ensure that Java specifications are able to be independently implemented and distributed under open source licenses.

We are grateful for the strong support from the community, and believe it is a validation of the work the ASF is doing in the JCP. Our efforts to transform the JCP into a truly open specification ecosystem help strengthen the value of Java for everyone -- for implementors of open source projects such as those found at the ASF and elsewhere, for students, educators and academics using Java for teaching and research, for independent software vendors that build innovative products and services on Java, and for commercial users in all areas of economic activity that depend on Java to run and grow their businesses.

Through the JSPA, the agreement under which both Oracle and the ASF participate in the JCP, the ASF has been entitled to a license for the test kit for Java SE (the "TCK") that will allow the ASF to test and distribute a release of the Apache Harmony project under the Apache License. Oracle is violating their contractual obligation as set forth under the rules of the JCP by only offering a TCK license that imposes additional terms and conditions that are not compatible with open source or Free software licenses. The ASF believes that any specification lead that doesn't follow the JCP rules should not be able to participate as a member in good standing, and we have exercised our votes on JSRs -- our only real power on the JCP -- accordingly. We have voted against Sun starting and continuing JSRs, and have made it clear that we would vote against the JSR for Java SE 7 for these reasons.

In light of Oracle Corporation failing to uphold their responsibilities as a Specification Lead under the JSPA and breaking their signed covenants with the Apache Software Foundation that are the conditions under which we agreed to participate in the JCP, we call upon the Executive Committee of the JCP to continue its clear, strong and public support for Java as an open specification ecosystem that is a level playing field for participants in order to ensure that anyone -- any individual or commercial, academic or non-profit entity -- is able to implement and distribute Java specifications under terms of their choice. Specifically, we encourage the other members of the JCP EC to continue with their support of our position regarding Oracle, and vote accordingly on the upcoming Java SE 7 vote.

The ASF will terminate its relationship with the JCP if our rights as implementers of Java specifications are not upheld by the JCP Executive Committee to the limits of the EC's ability. The lack of active, strong and clear enforcement of those rights implies that the JSPA agreements are worthless, confirming that JCP specifications are nothing more than proprietary documentation.

I think there is little to add (I've reproduced verbatim). I do hope that Oracle take one last look at what is about to happen here and offer some form of meaningful compromise. It may be too late, but the impact of this is potentially serious.

This new statement appears to superceed the previous statement.

For the record, although I am an Apache member, I was not party to the specific decision making in either statement.

Stephen Colebourne
Apache Software Foundation member, speaking personally
Oracle Java Champion, speaking personally
Not a committer on Harmony or OpenJDK

View the privacy statement