motlin.com

Bottom types, or “Why would Java be better off without null?”

Craig P. Motlin — Wed, 06 Jun 2012 03:14:08 +0000

When I first heard the idea that Java would be better off without null, I looked around for an explanation I could understand. Unfortunately, everything I found used code examples from languages without null. If I knew those languages, I wouldn’t be searching.

I wound up learning the “hard way” but now I figure I can answer the question using Java examples. Let’s jump right in.

Code Example 1

A and B are classes, not interfaces, so no Java class can extend both. There’s no way the method() can return a meaningful answer. It could return null, but not a real object. So let’s rename method() to undefined().

Think about how you might implement undefined(). Is it even possible to get this code to compile? If you replaced the method calls with null, it would obviously work.

But methods can’t say they return null.

We’d need a type there instead. Ok, so what is the type of null? Let’s look at another example.

Code Example 2

Here we’re using the same code “throw new RuntimeException()” in two contexts that expect two incompatible types. Ok, so what is the type of “throw new RuntimeException()” ?

The answer is: they’re both a form of the bottom type, which is logically a subtype of all other types. It’s the logical opposite of Object, the top of the Java hierarchy. It’s a fairly weak form of the bottom type without a name. Let’s say we enhance Java by giving it the name Nothing. This code would compile:

How can we fill in the implementation? We’ve already seen two ways: return null, or throw.

How else? We can recurse in an infinite loop.

How else? We can cast.

This code would compile but throw a ClassCastException at runtime, so it’s logically the same thing as throw new RuntimeException().

Of these choices, throwing is already really useful for pre-conditions. It’s idiomatic to call the method “error” when it takes a String parameter, and “undefined” without any parameters.

Since Java doesn’t have the Nothing type, we just use void today. In the following code example, Nothing is slightly more accurate than void, but it doesn’t really matter since the return type isn’t used.

But the point of a bottom type is you can return it.

This is pretty much what Scala has. Its bottom type is called Nothing. Haskell has a bottom type but leaves it unnamed. To see why, let’s look at my first question again.

Is it possible to get this code to compile? Yes, we can implement undefined() without the Nothing type, as follows.

Optionality

In Java, we use null for more than just indicating an error or from unreachable code. We also use null to represent optionality.

If map doesn’t contain key, value will be null. What about the reverse? Is it true that when value is null, map doesn’t contain key? Nope, because map may contain the entry (key -> null), in which case we need to remember to check map.containsKey(key).

In languages without null, we need another way to represent optionality.

Now methods that sometimes return null could be refactored to return Option instead. For example, Map.get(K) could return Option. Calling None.get() throws, which is logically equivalent to calling a method on null and getting a NullPointerException. If that were the whole story, Option wouldn’t be very useful, but it’s normal to add additional functionality. For example, Option.getOrElse(T default) could return Some.value for Some and default for None. Another example, forEach(block) could execute the code block on Some.value and just do nothing on None.

Putting it all together

The two concepts (bottom types and Options) come together in None.get(). Since it never returns a real value, it could return the bottom type. The fact that None is generic on type T and get() returns T makes the method look just like our earlier implementation of undefined(). With type variance, we could make None into a Singleton implementing Option.

Methods like Map.get() could return the Singleton.

With Option, we can get rid of null from the language completely. But should we? On the JVM, probably not. The Option wrapper objects waste a lot of memory. Let’s say we have a Person class with field private final Address workAddress which may or may not exist. Replacing the Address with Option

adds an extra 16 bytes per non-null workAddress (on a 64 bit JVM). Maybe this cost could be optimized away in a future version of the JVM but we’d pay it today.

However, there are big benefits of consistently using Option instead of null. Using the type system to reflect optionality communicates more about your API, plus it prevents common mistakes. It trades one kind of syntax cruft for another. I think it’s a clear win in languages with lambdas and so Option will be more compelling with Java 8. You can see the difference with Scala today.

Scala and Haskell

Scala actually has two bottom types. Nothing works just like the examples we’ve seen so far.

Scala can even infer Nothing as the return type in this example.
def undefined: Nothing = throw new RuntimeException def undefined = throw new RuntimeException

Scala has a second bottom type called Null.

def undefined: Null = null

Why? Primitives.

So Nothing and Null are both bottom types in Scala, and Nothing is “below” Null. In the Java version public static T undefined() we can’t differentiate between non-normal termination and null. But in Scala we can, because it’s a compiler error to return null from a method declared to return Nothing.

Haskell doesn’t have this problem because it doesn’t have any of the complicating factors we’ve gone over. Haskell doesn’t have type casts, primitives, or null! That means the bottom type can only recurse infinitely or throw. Without null, there’s no need for two bottom types. In fact, Scala only names Nothing to differentiate it from Null. Without two bottom types, Haskell gets by without naming the bottom type (using the generics trick alone).

a is a generic type like Java’s .
[Char] is a list of Char, in other words, a String.

So in Haskell, the method signature undefined :: a is enough to know with certainty that the body doesn’t return a value. Haskell’s type system is very strict and precise, so there are many examples like this where the type signature alone tells you everything you need to know about the function.

Future versions of Java will never eliminate null, but remember that Option is a good option.

Maven Best Practices

Craig P. Motlin — Sat, 20 Aug 2011 15:25:36 +0000

Always use the maven-enforcer-plugin

Enforce dependency convergence

Otherwise it’s possible that you depend on two different jars which both depend on log4j. Which one gets used at compile time depends on a set of rules that you shouldn’t have to remember. They can both (!) get exported as transitive dependencies.

Require plugin versions (for all plugins, even the built in ones)

Define them in pluginManagement in the parent pom to define versions

Otherwise a new version of maven-surefire-plugin could break your build

Use dependencyManagement in the parent pom to use versions consistently across all modules
Periodically run mvn dependency:analyze

It’s possible that you’re getting a dependency transitively that you directly depend on at compile time. If so, it’s important to add it to your pom with the version you require. This plays nicely with the enforcer plugin.
It’s possible that you’re declaring extra dependencies that you don’t use. This doesn’t work properly 100% of the time, especially with libraries that are designed to have optional pieces (i.e. slf4j-api gets detected properly, but slf4j-log4j12 fails).

Maven CI best practices http://www.sonatype.com/people/2009/01/maven-continuous-integration-best-practices/

Automate Snapshot Deployment through the CI system
Isolate Local Repostitories

use -Dmaven.repo.local=%system.teamcity.build.workingDir%/.m2

Regularly purge local repositories
Enable Batch Mode with -B

This will make the logs shorter since it avoids the dependency download progress logging. It also ensures that the build won’t hang due to waiting for user input.

Enable Full Stack Traces with -e

Put the parent POM in the root directory, next to the child modules
Other resources

How to set up the rxvt terminal for Cygwin

Craig P. Motlin — Thu, 19 May 2011 04:23:08 +0000

Download and run the Cygwin setup file.

Select Install from Internet.

Enter C:\cygwin as the Root Directory.

When you get to the package list, search for rxvt and click the word skip so that it changes to a version number.

Add any other packages you want. I always install at least these:

git
git-completion
git-gui
git-svn
gitk
openssh
rsync
rxvt
screen
tig
unzip
vim
wget

Create a new file called C:\cygwin\rxvt.bat and add these contents.

Replace “Craig” with your user name. If you don’t like large fonts or Lucida Console, you can change that as well.

Create a desktop shortcut to C:\cygwin\rxvt.bat and change its icon to C:\cygwin\bin\rxvt.exe.

The default colors aren’t that great. You can copy my .bashrc, .dircolors, and .Xdefaults from my dotfiles github repository or you can create your own.

How to do Version Numbers

Craig P. Motlin — Sun, 28 Nov 2010 04:14:39 +0000

Motivation

Why do we bother with multi-part numbers at all? Why not just use use a single revision number, like version 123? The parts of a version number communicate the magnitude of the change between the two versions. In the case of a library, this translates to how difficult the upgrade will be, which in turn tells about the level of source, binary, and serialization compatibility between the two releases.

Setting the right level of compatibility between releases is about striking a balance. Upgrades that are fully source and binary compatible are much easier to take for the client, but much more work for the authors. When releasing frameworks I prefer to use a three part version number of the form major.minor.maintenance. Bug fix (maintenance) releases must be source, binary, and serialization compatible. Minor releases ought to be binary and serialization compatible. Major releases need not be compatible.

Fix

When we find a bug, it’s often serious enough that we want clients to upgrade immediately. We don’t want to give them any excuse to avoid it. That means we ought to preserve binary, source, and serialization compatibility. That way the new version can be a drop in replacement, meaning it will work without recompilation.

The fact that a library author bothered to produce a minor version number communicates intent to the client. The author went through the effort to create a fully compatible release in order to make your life easier. The author believes the upgrade is important.

Occasionally fixing a bug will cause an incompatibility. It’s unfortunate but sometimes necessary. If it’s possible to limit the break to the piece of code that wasn’t working anyway, then it seems justified. Some work is required to get things working properly, it’s as simple as that.

Major

Incrementing the major version number indicates that backwards compatibility was broken. The new version can be binary incompatible, source incompatible, and the serialized form of its classes may have changed as well.

It’s like a last resort, but it’s an important one. Some changes are just too big. The authors will not always be able to or want to preserve compatibility. It’s important to schedule major releases so that there is balance. If you schedule them too frequently, clients will resist doing upgrades. Every major upgrade requires recompilation, maybe some development work, and testing. There’s little incentive to upgrade to any particular version at any particular time. It becomes very difficult to synchronize the version used across an entire organization. On the other hand, if you don’t create them frequently enough then clients may have to wait a very long time for certain features.

In my experience it’s good to create a major release roughly every six months.

Minor

Minor releases are the most complex because there are a few choices for what they can mean. Minor releases can be binary compatible with the previous release, source compatible, or both. There’s also the choice of whether it should indicate serialization compatibility as well. If you look at what open source projects do, you won’t find a lot of consistency.

I believe that minor version releases should be binary and serialization compatible with the previous release, but not necessarily source compatible. I’m assuming familiarity with binary and source compatibility. If not, you can read about it here.

Compatibility Trade-offs

Let’s dive into the trade-offs between work on the client side and work on the library developer side. Binary compatibility means that no coordinated releases across teams are necessary. Source compatibility means no development effort is required.

Why minor versions should be binary compatible

If your minor versions are binary compatible, it’s easier for everyone to upgrade. More importantly, teams who are slow to upgrade will not hold up other teams from taking the latest version.

Let’s say library B depends on version 1.0.0 of library A, and application C depends on both libraries. When binary compatible version 1.0.1 of library A is released, application C can start using it without involving anyone who works on library B. When a new version of a library is not binary compatible, it can take a very long time to propagate through an organization or through a software stack. With binary compatibility, no coordination across teams is necessary.

Why minor versions should not have to be source compatible

If a release is source compatible, it still needs to be recompiled but there won’t be any compiler errors requiring fixes. However, it doesn’t reduce the overall amount of work, it just pushes it later. If library designers think it’s a good idea to change API in some source breaking way, then they’ll do it some day and the clients will see those compiler errors eventually. Postponing just bunches them all up and makes major versions more painful.

In addition, source compatibility doesn’t affect what can get deployed. You might have to stick with the old jar at compile time to avoid compiler errors, and yet you can still deploy with the new jar if it’s binary compatible.

Making minor version numbers source compatible adds restrictions and requires more work from the library developers without giving clients much benefit. Therefore I recommend that minor versions are binary compatible but not source compatible. This might not be what clients are expecting though so it’s important to communicate the compatibility policy clearly.

Binary and Source Backwards Compatibility

Craig P. Motlin — Fri, 26 Nov 2010 16:42:37 +0000

It’s pretty easy to break binary and source compatibility simultaneously. Remove one parameter from a public method or delete a method entirely and no existing code that used the method will compile or run. But how can we break just binary or just source compatibility but not both? Let’s look at examples of each in Java.

Source Compatibility

This methods prints out every item in a collection.

public void printAllElements(Collection collection)
{
  for (Object each : collection)
  {
    System.out.println(each);
  }
}

The method takes a Collection as its parameter, but it could really operate on any Iterable. So let’s make that change:

public void printAllElements(Iterable iterable)
{
  for (Object each : iterable)
  {
    System.out.println(each);
  }
}

This change is source compatible, meaning all source that compiled against the previous version of this code will still compile. That makes sense, because every Collection is an Iterable, so every existing reference to this method is still valid.

This change is not binary compatible though. If you swap out the old jar for the new one without recompiling, you’ll get an error at runtime like this:

Exception in thread “main” java.lang.NoSuchMethodError: com.motlin.Printer.printAllElements(Ljava/util/Collection;)V

The bytecode of the client application refers to this method very specifically. You can see the method signature in the error message, and the type of the parameter is part of the signature. The method com.motlin.Printer.printAllElements(Ljava/util/Collection;)V no longer exists. To use the new library, client code must be recompiled. That might not sound so bad, but transitive dependencies need to be recompiled as well. That means if the client application uses a second framework which depends on the first, the second framework will also need to be recompiled. In a large dependency graph, the extra work can add up.

Binary Compatibility

How can we break source compatibility while preserving binary compatibility? One way is to add a method to an interface. Let’s write a tiny library with one interface and one concrete implementation. Here’s the complete source of version 1.

public interface MyInterface
{
  void doSomething();
}

public class MyImpl implements MyInterface
{
  public void doSomething()
  {
    System.out.println("");
  }
}

Version 2 adds a second method to MyInterface called doSomethingElse(). That’s not a source compatible change. MyInterface is public, which means any client can write their own implementations of it. When clients upgrade, their implementations won’t implement doSomethingElse() yet so they’ll get a compiler error like this:

com.motlin.MyImpl is not abstract and does not override abstract method doSomethingElse() in com.motlin.MyInterface

The rules for binary compatibility are quite complex and you can read the rules elsewhere. The reason why this change is binary compatible is analogous to why the previous change wasn’t. A client application compiled against version 1 has references in the bytecode to the method doSomething() which is void and takes no parameters. That method is still there in version 2, so it will work, simple as that. Internally, doSomething() can call doSomethingElse() in version 2 and it would still be ok. The internal implementation of a method does not change its signature, so everything is still fine. Binary compatibility is very dynamic. It is determined at runtime and difficult to reason about earlier.

Tabs vs. Spaces

Craig P. Motlin — Thu, 04 Nov 2010 12:18:21 +0000

… and why I can’t just let it go.

Before I say which is better, I want to explain why I think the issue is even important. Whitespace is ignored by the compiler so it’s purely for readability. It’s there for you. Variable names have meaning and describe the intention of code. Similarly, whitespace describes the structure of a program.

Suppose there is a hypothetical programmer who is intelligent and writes correct programs, but he names his variables things like “var1″ and “var2.” In real life, most code needs to be modified at some point, through no fault of the original programmer. Requirements change. When another developer (or even the original programmer) needs to read this code, he will find it utterly useless. He will waste lots of time trying to understand it. He may even re-implement some existing functionality simply because he couldn’t find it buried there in the obfuscated code. I would go so far as to say this hypothetical programmer deserves to be fired simply for his unusual naming style.

Code with inconsistent indentation isn’t so useless – fortunately for us it is easily repaired by automatic formatting tools. However, leaving a file with inconsistent indention puts that burden on next person who reads it. The next reader will take slightly longer to mentally parse the file since he must figure out scope information which the indentation level could have communicated. And someone will eventually need to fix it, manually or with a code beautifier.

On the one hand, I feel like a programmer who can’t indent his source consistently also can’t be trusted to make the proper design and engineering decisions made while implementing even the smallest feature. In reality, I know that most code is already not indented consistently. Even if it was, accidents do happen. A merge tool or a terminal editor may easily cause inconsistencies that would have never happened in the IDE, for example. Thus to truly have a standard, it must be enforced externally – by the IDE on check-in, or better yet by the version control system.

Now the controversial part: Tabs are better than spaces.

There – I’ve said it. Here’s why. With either choice, a standard is being imposed on all the developers who work on the project. But choosing spaces really forces two separate things – the ASCII character used for indentation plus the width of the indentation. Choosing tabs only imposes the ASCII character used for indentation, which is the absolute minimum needed to have a standard.

A common argument is that if you use spaces, it looks the same in any editor without doing any extra configuration. But in order to have a standard, you must do extra configuration. Otherwise the first time you hit the tab key, you may introduce an inconsistency. If you use vim with the default configuration, you just inserted a tab. Emacs is worse – it inserts a mix of tabs and spaces by default. And I’m sure another editor inserts spaces by default.

An alternate suggestion is to never use the tab key, but that can’t be enforced. It shouldn’t be anyway because tapping the space bar is a waste of time. Like I said, a real standard must be enforced by the version control system. As long as a standard is being imposed, it should be tabbed indentation so that each teammate may configure his editor to use the width he prefers. Each teammate must configure his editor, or else waste more time fixing the whitespace when the version control system rejects his check-ins.

There is a second argument in favor of tabs that is extremely simple. Have you ever seen a file that was indented with 4 spaces in some parts and 2 spaces in other parts? Two coders who use tabs could never interfere with each other in this way.

Risk Calculator

Craig P. Motlin — Mon, 08 Dec 2008 03:00:59 +0000

Run

Launch Risk Calculator

Additional Information

Subversion repository
Javadoc
Old releases

Configuration (dot) files

Craig P. Motlin — Sun, 07 Sep 2008 17:09:39 +0000

These are files I used to configure my command line environment, plus the scripts used to generate these pages. These files are always evolving. You can find the latest versions on github.

Configuration files

.alias
Aliases used by bash and zsh
.bashrc
For GNU BASH
.profile
Brings me directly to bash if ksh is my default shell
.screenrc
For GNU Screen
.vimrc
For Vim
.zshrc
For Zsh

Ant Timer

Craig P. Motlin — Sun, 07 Sep 2008 16:17:41 +0000

Download

ant-timer.jar Compiled classes
ant-timer-src.zip Source code

Additional Information

Subversion repository
Javadoc
Old releases

Five minute tutorial

The Ant Timer is a plug-in that prints out diagnostic messages when ant targets and tasks take longer than the configured amount of time. You can use it with any ant build. Here is some sample output when I use it on my renamer program.

javadocs: [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source files for package com.motlin.base... [javadoc] Loading source files for package com.motlin.base.log4j... [javadoc] Loading source files for package com.motlin.renamer... [javadoc] Loading source files for package com.motlin.renamer.exceptions... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.6.0_06 [javadoc] Building tree for all the packages and classes... [javadoc] C:\cygwin\home\Administrator\dev\renamer\trunk\src\com\motlin\base\log4j\StdOutErrAppender.java:318: warning - Tag @link: reference not found: org.apache.log4j.net.SocketAppender [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... [javadoc] Generating C:\cygwin\home\Administrator\dev\renamer\trunk\dist\api\stylesheet.css... [javadoc] 1 warning XXX task [renamer.javadocs.javadoc] took 2 seconds to run XXX task [renamer.null.sequential] took 2 seconds to run XXX task [renamer.javadocs.ac:outofdate] took 2 seconds to run XXX target [renamer.javadocs] took 2 seconds to run

BUILD SUCCESSFUL
Total time: 2 seconds

Running ant with the custom listener is pretty simple. Here’s the command that produced the output above.

ant -listener com.motlin.ant.timer.TimerListener -lib ant-timer-1.0.jar -lib lib/joda-time-1.5.2.jar -Dtarget.timer=2 -Dtask.timer=1 -Dtimer.prefix="XXX" clean javadocs

Let’s break it down.

-listener com.motlin.ant.timer.TimerListener

This is the class name of the custom task.

-lib ant-timer-1.0.jar -lib lib/joda-time-1.5.2.jar

Unfortunately, the Ant Timer depends on joda time. It didn’t have to but I like it better than using the built in classes for working with times. You’ll need to download the joda jar separately.

-Dtarget.timer=2 -Dtask.timer=1 -Dtimer.prefix="XXX"

You can configure the behavior through these properties. The first two are the timeouts in seconds for targets and tasks. If you leave them off they default to 10 seconds and 1 second respectively. The prefix makes it easy to find the output printed by the custom listener. If you leave it off it defaults to “!!!”

Universal toString()

Craig P. Motlin — Sat, 06 Sep 2008 15:34:34 +0000

Download

UniversalToString.jar Compiled classes
UniversalToString-src.zip Source code

Additional Information

Subversion repository
Javadoc
Old releases

Five minute tutorial

UniversalToString is a java class that prints out the internal state of an object using reflection. The UniversalToString class builds a String by recursively traversing the contents of an object and printing out the names, types, and values of all of its fields. It is loosely based on some code in Core Java 2, Volume I – Fundamentals. The simplest way to use the UniversalToString class is in the implementation of your own classes’ toString methods. Say we have a simple Person class:

public class Person {
    private final String first;
    private final String last;
    public Person(final String first, final String last) {
        this.first = first;
        this.last = last;
    }
}

Person does not implement the toString method, so when it is printed to standard out, Object.toString is used instead. The generated String contains the type name and the address in memory where the object lives. Normally this is not very useful information.

Person john = new Person("John", "Smith");
System.out.println(john);

Console output:

tostring.Person@19821f

To improve the output, we can override the toString method in the Person class.

@Override
public String toString() {
    return "Person: first [" + this.first + "] last [" + this.last + "]";
}

Person: first [John] last [Smith]

Implementing the toString method like this is tedious, repetitive, and fragile. That’s where UniversalToString comes in.

@Override
public String toString() {
    return UniversalToString.print(this);
}

Person(first=John,last=Smith)

Collections

UniversalToString has special logic for arrays, collections, and maps. This example uses a list, but the output is similar for each.

Person john = new Person("John", "Smith");
Person jane = new Person(“Jane”, “Doe”);
List list = new ArrayList();
list.add(john);
list.add(jane);
System.out.println(UniversalToString.print(list));

ArrayList{Person(first=John,last=Smith), Person(first=Jane,last=Doe)}

Custom formatting

Unfortunately our work isn’t done. The String created by UniversalToString isn’t particularly informative for certain objects. For example, let’s change the last example to print one Person and one Date:

Person john = new Person("John", "Smith");
Date date = new Date(1206552944822l);
List list = new ArrayList();
list.add(john);
list.add(date);
System.out.println(UniversalToString.print(list));

ArrayList{Person(first=John,last=Smith), Date(fastTime=1206552944822,cdate=null)}

We can customize the formatting of UniversalToString by creating an implementation of TypedToString that handles Date objects.

public class DateToString implements TypedToString {
    private final DateFormat dateFormat;
    public DateToString() {
        this.dateFormat = new SimpleDateFormat();
        this.dateFormat.setTimeZone(TimeZone.getTimeZone("America/New_York"));
    }
    public boolean handlesType(final Object object) {
        return object instanceof Date;
    }
    public String print(final Object object) {
        return this.dateFormat.format((Date) object);
    }
}

Now we just need to change the line that prints the list.

System.out.println(UniversalToString.print(list, new DateToString()));

ArrayList{Person(first=John,last=Smith), 3/26/08 1:35 PM}

The second argument to print is varargs TypedToString, so we can pass in as many special formatting rules as we need.

motlin.com

Bottom types, or “Why would Java be better off without null?”

Code Example 1

Code Example 2

Optionality

Putting it all together

Scala and Haskell

Maven Best Practices

How to set up the rxvt terminal for Cygwin

How to do Version Numbers

Motivation

Fix

Major

Minor

Compatibility Trade-offs

Why minor versions should be binary compatible

Why minor versions should not have to be source compatible

Binary and Source Backwards Compatibility

Source Compatibility

Binary Compatibility

Tabs vs. Spaces

… and why I can’t just let it go.

Risk Calculator

Run

Additional Information

Configuration (dot) files

Configuration files

Meta

Ant Timer

Download

Additional Information

Five minute tutorial

Universal toString()

Download

Additional Information

Five minute tutorial

Collections

Custom formatting