Never, ever use Iterable as a return type

by Алекс Руис on September 6, 2012

This is a short rant. Please do not use Iterable as a return type. Its Javadoc states that

Implementing this interface allows an object to be the target of the “foreach” statement.

That’s the only purpose of this interface.

Other than that, this interface is almost useless, or at least a PITA to consume. For example, to obtain the size of an Iterable you’d need to do this:

Iterable<String> names = getRecentUsers();
int size = 0;
for (String name : names) {
  size++;
}

Or, just to check that an Iterable is not empty:

Iterable<String> names = getRecentUsers();
boolean empty = !names.iterator().hasNext();

Horrible, isn’t it?

Return a Collection instead. If your users may need to access to the elements by index, return a List.

(I actually think that Iterable should not even used for parameters, but I’m not 100% sure yet.)

Update: After reading the comments in this post, I realized I didn’t cover one use case: collections that contain a big number of elements and are lazily loaded. I also did not specify the context I was talking about: domain-specific APIs that you are in control of.

I still think that Iterable is a poor choice. Since we are in control of the API we can provide something more useful (or potentially more useful) to the consumers of such API. In the case of lazily-loaded collections, provide something like a Page object. It may or may not implement Iterable. A Page class may contain other useful information besides simple iteration of elements (e.g. the page number.)

Changing the subject a little bit, this is something that I’ve seen quite frequently, all in the name of “flexibility”:

Iterable<String> getRecentUsers() {
  List<String> users = fetchFromDb();
  return Collections.unmodifiableList(users);
}

Since we are already dealing with List, why not just return the same type?

Update 2: If you are thinking about Guava‘s usage of Iterable. please read the comments, which includes one from the man himself, Kevin Bourrillion.

Update 3: This is my position about general-purpose libraries like Guava:

Guava’s goal is to make Java easier to use. It provides a lot of useful functionality that IMHO should be in the JDK itself. As such, it needs to deal with very general classes such as Iterable. Plus, it provides utilities to overcome some of Iterable‘s deficiencies (like lack of a isEmpty method.)

My post is about domain-specific APIs that you provide to your users, not something like Guava.

{ 28 comments… read them below or add one }

Wim Deblauwe September 6, 2012 at 3:35 am

I do believe using Iterable can be needed sometimes, e.g. If the collection would not fit in memory. I believe Google Guava uses it for that reason. But indeed, most of time, it is a PITA.

Reply

Alex Ruiz September 6, 2012 at 9:36 am

Hi Wim,

I failed to see all the comments that were up for moderation and responded to one that came after yours (the one from Sebastian.) I updated the blog post with my opinion regarding big collections.

Cheers,
-Alex

Reply

Woidda September 6, 2012 at 3:43 am

So you think for example
http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/base/Splitter.html#split(java.lang.CharSequence)

is a bad design?

You can use http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/Iterables.html#isEmpty(java.lang.Iterable) for checking if an Iterable is empty.

There are some cases where an Iterable is very handy (such as where a library can use lazy initialization for optimizations). But I agree: Most of the time a Collection is the right return type.

Reply

Alex Ruiz September 6, 2012 at 7:27 am

Guava, a wonderful library that makes Java a lot easier to use, is a general-purpose library. Probably it is as general-purpose as the JDK itself. I’m not sure that in the case of Splitter.split(CharSequence) there are no other choices though. IMHO, the Iterables class exists to supplement the deficiencies of Iterable.

The opinion I expressed in the post was geared towards designing your API for the domain you are working on, which is way more specific than Guava’s.

Reply

Matthew September 6, 2012 at 10:40 am

> The opinion I expressed in the post was geared towards designing your API for the domain you are working on, which is way more specific than Guava’s.

That’s not the impression the title of your post gives.

Reply

Alex Ruiz September 6, 2012 at 10:43 am

Fine. Would you like me to change it to: “Never, ever use Iterable as a return type when you are in control of your domain-specific API”?

Reply

Kevin Bourrillion September 6, 2012 at 10:44 am

I’m not (yet) taking a position on the general debate but I will say this.

I may not have written Splitter.split(), but I certainly approved its API design. It has indeed been one of the biggest mistakes we’ve made. We desperately wish to fix it to return List, but change is not easy now.

And, as far as I can see, it is the only example in all of Guava where we return Iterable (apart from the methods which take _in_ Iterables and return some tweaked view of the same thing).

Reply

kondi September 6, 2012 at 3:43 am

I completely disagree…
If you need the size as well you can consider returning a Collection instead, but many times the Iterable is enough, it is more generic, exposes less and they can be lazy.
It sounds really creazy to return a Collection instead of Iterable just to be able to check it easier if it is empty.

You have a set of nice utilities to handle them in guava: http://docs.guava-libraries.googlecode.com/git-history/release/javadoc/com/google/common/collect/Iterables.html (for example Iterables.isEmpty)

Reply

DR September 6, 2012 at 3:45 am

I *really* disagree with this. Iterable represents an object that behaves like a forward-only cursor. It is intended to identify objects which are not proper collections, or collections for which the bounds are not known at the time it’s built. The fact that you can’t call size() on it is quite deliberate.

Now, of course, if you have an object that behaves exactly like a Collection, by all means return Collection. But that is not every case, and Iterable completely makes sense as a return type for something that behaves like, for example, a forward-only database result set.

Reply

Gerard Davison September 6, 2012 at 3:46 am

I agree that generally Iterable is a PIA as a return type; but I feel the kittens are safe in situations where the “collection” you are reading over has an indeterminate content.

Take for example the method getJavaFileObjects from the Java API:

http://docs.oracle.com/javase/7/docs/api/javax/tools/StandardJavaFileManager.html#getJavaFileObjects(java.lang.String…)

As you know this might involve searching multiple source paths and many many jars and the user might just be happy with the first hit. So the API doesn’t want to expose the ability to query the possible size with a Collection as that would be inefficient; but instead returns a Iterable which only care about the next entry.

Now Iterable as a parameter is another story; but does make adapting object objects without having to copy them into a collection easier. It is a pain that the collections API doesn’t consume Iterable instances more cleanly; but the Google collections API Iterables resolves this to some extent:

http://google-collections.googlecode.com/svn/trunk/javadoc/com/google/common/collect/Iterables.html

This class also has the “reverse” function which makes many for loops just that much more readable.

Of course as every other opinions are available,

:-)

Gerard

Reply

Alex Ruiz September 6, 2012 at 11:03 am

Hey Gerard, my dear friend!

I updated the post after reading some comments, to explain, in a better way, where I’m coming from.

I’m almost sure that example regarding StandardJavaFileManager is valid. The JDK provides general-purpose functionality (just like what I said about Guava.) My post was geared toward more domain-specific API.

Cheers,
-Alex

Reply

Mikael September 6, 2012 at 3:54 am

I disagree with you.

Saying to not use Iterable as return type or parameter is as if your were suggesting to only return ByteArrayInputStream instead of InputStream. The intent when returning an Iterable could be “be aware that iterating on the returned object is expensive and should be done only once, if you really need to iterate more than once, you should create a copy in a Collection”. Most of Java collection implementations do not have a copy constructor taking an Iterable but Google Guava Collection Utilities classes has (https://code.google.com/p/guava-libraries/wiki/CollectionUtilitiesExplained)

Allowing “foreach” is not the only purpose of this interface. It makes lazy collection filtering / transformation possible. You can chain function and/or predicate object and only apply them when iterating on the final Iterable ! Moreover, with suitable library (e.g. Google Guava with Iterables/Iterators utility classes), it is perfectly easy to use. For more on how to use functional idioms with Guava, have a look on this page https://code.google.com/p/guava-libraries/wiki/FunctionalExplained

Regards,

Reply

Sebastian September 6, 2012 at 4:17 am

Hi Alex,

generally I agree that Iterable is not the most convenient data structure on earth. Nevertheless I think that your example lacks an important detail. The method getRecentUsers could – theoretically – return a great number of users (depending on the meaning of ‘recent’ and the load of your application). So instead of loading all this information at once in order to allow answers to the ‘size()’ of the iterable, a client may want to display only 10 of them. So the implementation of ‘getRecentUsers’ could be lazy in a way that only a certain number of items is returned and only if a client is interested in more (Iterator#hasNext) the next batch of recent users is computed.

So I agree that a list is much more convenient from a client’s perspective, it’s way easier to implement a lazy iterable instead of a lazy list so therefore there are many good reasons to offer an Iterable as a return type. Of course the same holds for parameters.

Sebastian

Reply

Alex Ruiz September 6, 2012 at 7:03 am

Hi Sebastian,

I agree, the method getRecentUsers can return a big number of users and returning a Collection may be not practical. I’d say that still Iterable is not the solution. It does not provide anything useful other than iteration. A better solution could be to return something like a Page object. Having a Page class (which may implement Iterable) gives you the flexibility to add more functionality that consumers may need.

-Alex

Reply

Peter September 6, 2012 at 6:11 am

Alex, Iterable is actually a perfectly good choice for a return type in the right context :-) For instance, you have a stream of objects that you want to give a one-time access to (you parse each object from the stream as you iterate). At the time you return the Iterable, you don’t know if there’s anything in the stream and to find out you’d have to read from it. You don’t know how many objects there are either. A higher-level API could very well wrap that Iterable in a List or Collection, but then they have to take care of buffering, etc.

Reply

Alex Ruiz September 6, 2012 at 7:30 am

Peter, I think if when we are in control of the API, we can make our users a big favor and give them something richer or potentially richer than Iterable, like I mentioned in the reply to Sebastian, we can provide something like a Page object (page from pagination, nothing related to HTML.)

Reply

Matthew September 6, 2012 at 6:23 am

Google Guava libraries. That is all. http://code.google.com/p/guava-libraries

Guava makes extensive use of iterables. Check it out, it may change your perspective.

Reply

Alex Ruiz September 6, 2012 at 7:31 am

Matthew, please read my reply to Woidda :)

Reply

Matthew September 6, 2012 at 6:28 am

I would like to add that null literals are the devil’s method arguments.

Reply

Jens September 7, 2012 at 12:59 am

I agree with you on the problems regarding missing getSize() and isEmpty() methods of the interface Iterable. However—and I would say this is almost independent from the specific context—using Iterable for (input) parameters of a method is a good choice if the method does not need these two methods. Depending on the stability of the code, I would say to return the most specific classifier as possible. That is, if a method could return a List instead of a Collection, I would return the List.

Anyway, IMHO there is a kind of design gap between Iterable and Collection, and I would figure that this is the real problem: there is no immutable interface for collections in the JDK. Iterable is immutable, but it misses some important methods. Collection is mutable, which is often a problem, e.g., in case of pseudo collections which are created on the fly or composed or derived (filtered) from other collections (I often use that pattern).

Reply

Alex Ruiz September 7, 2012 at 11:45 am

Jens, I’m with you. The only solution for the immutability problem is, IMHO, Guava.

Reply

Frank Kieviet September 7, 2012 at 11:16 am

The arguments for using Iterable for lazy loading are not very strong. The biggest problem is that Iterable does not have a close method, which makes lazy loading from any resource that needs to be closed, problematic. Also, most operations that involve reading from a resource throw checked exceptions, which makes the use of iterators awkward.

For that matter, Collection doesn’t do much better in that respect either.

Reply

Alex Ruiz September 7, 2012 at 11:43 am

Thanks Frank!

I completely agree. Lazy-loading is a completely different animal. The collection interfaces are not suitable for this scenario for the reasons you mentioned, plus there may be some domain-specific ones. A custom class that takes in consideration all these aspects is a better solution.

Cheers!

Reply

ap September 7, 2012 at 7:58 pm

Hallo,

i would like to agree with you about the state of the Iterable and actually it is incomplete. I would like also point here that most of standard Java collections are incomplete as well. Returning a naked Java Iterable is really bad idea in all circumstances and anyone who thinks it’s good – should be fired immediately. However, returning naked Collection interface are too versatile and might cause more troubles than it’s at the sketchy view. But, you still can use upper-bound and lower-bound facilities for that, to actually restrict a sense of the returning type. No matter that it is, an Iterable wrapper could be a good solution and it may contain methods to wrap this wrapper into *particular collection* of your own choice. And here is how:

public final class IterableW<A> implements Iterable<A> {
  private final Iterable<A> i;

  private IterableW(final Iterable<A> i) {
    this.i = i;
  }

  /**
   * You can wrap over it. Wraps the given iterable.
   */
  public static <A> IterableW<A> wrap(final Iterable<A> a) {
    return new IterableW<A>(a);
  }

  /**
   * You can even wrap it with a function.
   * Provides a function that wraps the given iterable. F is ommited.
   */
  public static <A, T extends Iterable<A>> F<T, IterableW<A>> wrap() {
    return new F<T, IterableW<A>>() {
      public IterableW<A> f(final T a) {
        return wrap(a);
      }
    };
  }

  /**
   * Returns an Iterable that completely preserves the argument.
   * The unit function for Iterables.
   */
  public static <A> IterableW<A> iterable(final A a) {
    return wrap(some(a));
  }

  /**
   * Yes, you even can deal with this.
   * Returns an iterator for this iterable.
   */
  public Iterator<A> iterator() {
      return i.iterator();
  }

  /**
   * You can go even further with this wrapper.
   * Returns a java.util.List implementation for this iterable.
   * The returned list cannot be modified.
   */
  public List<A> toStandardList() {
    return new List<A>() {

      public int size() {  } // readers exercise

      public boolean isEmpty() { } // readers exercise

      @SuppressWarnings({"unchecked"})
      public boolean contains(final Object o) { } // readers exercise

      public Iterator<A> iterator() { } // readers exercise

      public Object[] toArray() { } // readers exercise

      @SuppressWarnings({"SuspiciousToArrayCall"})
      public  T[] toArray(final T[] a) { }

      public boolean add(final A a) { }

      public boolean remove(final Object o) { }

      public boolean containsAll(final Collection c) { }

      public boolean addAll(final Collection c) { }

      public boolean addAll(final int index, final Collection c) { }

      public boolean removeAll(final Collection c) { }

      public boolean retainAll(final Collection c) { }

      public void clear() {
        throw new UnsupportedOperationException("Modifying an immutable List.");
      }

      public A get(final int index) { }

      public int indexOf(final Object o) {
        int i = -1;
        for (final A a : IterableW.this) {
          i++;
          if (a.equals(o))
            return i;
        }
        return i;
      }

      public int lastIndexOf(final Object o) {
        int i = -1;
        int last = -1;
        for (final A a : IterableW.this) {
          i++;
          if (a.equals(o))
            last = i;
        }
        return last;
      }
    }
}

Also, i would like to point on the lazy-loading of the collections — there is no such thing in Java. It doesn’t matter that is the return type of your method, it’ll always return something. Iterable or Collection — it’ll always return a particular instance of the class (concrete or anonymous). A laziness mean — that you declaring a type or a class in your code and it will be instantiated (and will consume memory) *only* when you actually accessing their reference. Consider some Scala code:

class DBType(record: DBRecord) {
  lazy val driver  = Class.forName(record.driver).newInstance().asInstanceOf[Driver]
  lazy val adapter = record.adapter()
}

Here values driver & adapter will be instantiated only when you’ll access them; not at the time of the DBType instance creation. If you’ll never access them — they wouldn’t be instantiated at all, hence will not consume memory. A good example of the *really* lazy collection in Scala are an immutable Stream element of the Scala collections -> http://www.scala-lang.org/api/current/index.html#scala.collection.immutable.Stream. It collects elements & not instantiates them; elements in the Stream will be instantiated only when you’ll access them. Otherwise only references exists. Welcome to Scala!

I hope something from that will help. Hey Alex, i wrote a blog post inside of your blog post, which might be called “a blog post injection” :)

Reply

ap September 7, 2012 at 8:07 pm

Sorry, the code above has been truncated by WP….. Here is more complete implementation -> http://hpaste.org/74432

Reply

Kevin Cline March 15, 2013 at 2:48 pm

I completely disagree. There is a run-time cost in converting an Iterable to a collection. Why should all clients of an API be forced to bear that cost? With Guava, it’s trivial to transform an Iterable to a Collection.

I would argue the opposite: unnecessarily returning definite collection classes like List leads to future pain when the internal implementation changes.

I would further argue that the return type of Splitter.split is correct, but the input type (CharSequence) is not. Why should Splitter not handle streams of characters, returning one substring at a time indefinitely?

Reply

Rob October 11, 2013 at 2:24 am

From the example Alex gives, he’s arguing against things like transforming Lists to Iterables, so you’ve already borne the run-time cost. You’re just restricting what the API user can do by turning a List into an Iterable; they can already call all the Iterable methods (by which I mean method: iterator()) if you give them a List.

Also, List (like Collection, and Iterable) is an interface, not a “definite class”. The internal implementation of which you speak is only related to any class that implements the List interface.

Reply

Daniel Kolman August 19, 2013 at 2:00 am

You Java guys should really learn more about C# and it’s “yield” keyword. Iterables (=Enumerables) are really the building stone for C#’s great LINQ, deferred evaluation and function composition. There are plenty examples on the web, e.g. http://xosfaere.wordpress.com/2010/03/21/lazy-evaluation-in-csharp/

Reply

Leave a Comment

Previous post:

Next post: