Android, Gradle & IDE Integration

by Алекс Руис on June 4, 2013

In this post I’m describing how we use the new “custom model” feature, introduced in Gradle 1.6, to provide IDE integration for Android projects that use the new Gradle-based build system. This feature was designed to allow Gradle plug-in developers provide a custom representation of a Gradle project through the Gradle’s Tooling API.

This post is intended to be a “note to self,” to have it handy once we start implementing Gradle support for Android projects in the Android Development Tools for Eclipse (ADT.)

Background

Android has a new Gradle-based build system that makes it easier to accomplish tasks that were previously hard or impossible to do. We want build.gradle files to be the one and only source of truth. This means that building from the command line, an IDE or a continuous-integration server should always result in the same output.

Before Gradle 1.6, Gradle integration with Java IDEs (Eclipse & IDEA) was been taken care of by Gradle’s tooling API: it provided (and still provides) a IDE-specific representation of a Gradle project that was good enough for regular Java projects. Android’s Gradle plug-in contains a rich set of project-related information that is needed for IDE integration, but also Android-specific data (e.g. additional source folders, resource folders, etc.) Unfortunately, the pre-Gradle-1.6 tooling API model did not allow us to pass that information to IDEs.

To help us achieve our goal, the Gradle folks implemented the changes we needed. The custom model mechanism was released in Gradle 1.6. Thanks, guys!

Android’s Gradle Model

Instead of creating IDE-specific Android/Gradle models, we decided to have an IDE-agnostic representation of a Gradle project. This way, we have a single source of information that is easier to maintain. IDE integration will be implemented as a plug-in for each supported IDE (in our case, Eclipse and IDEA.) The main benefit of this approach is that we can release the Gradle plug-in independently of the IDE integration plug-ins. For example, we can ship a new version of the Eclipse plug-in that has several bug fixes, without affecting the Gradle side of Android.

The Android Gradle model is a rich and detailed representation of an Android project. The core of it is the concept of build variants. A build variant is a combination of build type (e.g. “debug” vs. “release”) and an application flavor (e.g. “free” vs. “paid”.) Of course, this is a very high-level explanation. To learn more about build variants, please read the Android Gradle build system docs.

Build variants are important for IDE integration because each build variant has its own project structure. For example, a build variant has its own output file (.aar file) and its own set of generated source directories. To set up an IDE project we need to know all the source folders, dependencies and outputs of the selected variant in an Android project.

IDE Integration

On one hand, build variants are a powerful concept that makes it easy to create multiple types of the same application. On the other hand, this new concept pushes the limits of what an IDE can do. IDEs were not designed to handle multiple versions of a same project (not their fault: probably this is the first time that such thing exists!) To properly use build variants in an IDE, some fundamental changes need to be made.

  • Display all variants and allow user to select the variant to work on.
    This is not really a “fundamental” change. An IDE needs to show all the available variants, the selected variant and allow the user to select a different variant to work on. In Android Studio we implemented this as a “Build Variants” tool window:
  • React to changes in build variant selection.
    As difficult as it is to implement this feature, it is not a fundamental change either. Once the user selects a different build variant, the IDE needs to react accordingly:

    1. adjust source directories: only mark as “source” the directories specified by the selected build variant
    2. adjust dependencies to other projects or 3rd. party libraries
    3. opened editors need to show contents that are relevant to the selected build variant
    4. point to the correct output file (.aar file) for launching and debugging
    5. trigger a project build

    All of this have to be done correctly and, of course, pronto.

  • Refactoring.
    This is where all the fundamental changes need to happen. Even though we haven’t fully explored the scope of changes in refactoring that build variants need, we have some ideas.

    Take class renaming for example. Let’s say we have class com.example.Class1 in the project’s default configuration (i.e. the code that doesn’t change regardless of the selected build variant.) Our sample project has 2 build variants and each have their own version of class com.example.Class2. Both versions of Class2 have a reference to Class1 (let’s say has a field of type Class1.)


    What should happen if I rename Class1 to MyClass?

    Yep, both versions of Class2 need to be included in the rename refactoring. The problem is that the IDE only knows about one Class: the one that comes from the selected variant. The IDE has no notion of all the other versions of that class. To make this refactoring work we need to let the IDE know about all the possible versions of a class or resource. That could mean making changes to how an IDE indexes files. This is a lot more complex than it sounds. I’m sure this is just the tip of the iceberg.

    May the force be with us.

Other simpler (but equally important) features that IDE integration should include are:

  • Create Grade-based Android projects
  • Import existing Gradle-based Android projects
  • Automatically detect changes in Gradle files and update the IDE project structure accordingly
  • Use Gradle as the only mechanism for building

Conclusion

The new Android Gradle build system is one of the best additions to an Android app developer’s toolbox. It makes it a lot easier to create different versions of the same app (not an uncommon use case.) IDE integration with the new build system is crucial, but not trivial to implement.

In this post I shared some ideas of what needs to be done to make it happen. Some of the points discussed in this post are already implemented in Android Studio. By the time you read this post, we should have started work on Eclipse support.

{ 10 comments… read them below or add one }

Cole Markham June 4, 2013 at 3:27 pm

Since you are collecting your thoughts here, I will contribute mine…

I would think that the best way to handle these variants in Eclipse is to actually create a separate Eclipse project for each. That way you get all of the refactoring tools to work properly across all variants without having to try to hack the way that the IDE indexes files. It would be much simpler to just hide the extra projects from the user (see Mylyn[1]). I have also heard talk of nested projects being supported by Eclipse. I’m sure the community would be very open to development in that area.

In general, I think Eclipse needs to be better about integrating with various build systems. No one should be using the IDE for building deployment-ready software, but yet all of the Wizards build projects that are geared toward that. Meanwhile most of the community has moved to Maven/Tycho for building Eclipse-based projects, but from what I have seen the integration with the IDE is poor. I would personally love to see an abstract means of integrating the IDE with the build system which could be leveraged by both Gradle and Maven.

Just my two cents,

Cole

[1] http://eclipse.org/mylyn

Reply

Dennis Doubleday June 5, 2013 at 7:22 am

Cole, that sounds like the right idea. It is easy to create projects that reference other projects. You could have a common core project and dependent projects with variant code.

Reply

Marius Kjeldahl July 11, 2013 at 11:41 pm

Do you have any pointers to the updated version of the android gradle plugin source? I’m finding a repository at https://github.com/jvoegele/gradle-android-plugin, but that’s close to a year old now. I’m trying to add support for Kawa source code for building android apps and the souce code for the android gradle plugin would probably be a good place to start (considering my limited gradle knowledge at this point). Thanks.

Reply

Alex Ruiz July 20, 2013 at 6:54 am

You can find the source code of the Android Gradle plug-in at https://android.googlesource.com/platform/tools/build

Reply

Jan-Terje Sørensen July 16, 2013 at 12:21 am

Hi, are you guys also working on the possibility to do a `gradle eclipse` on Android projects to get the IDE project files created? Or do you know who I can contact?

Reply

Alex Ruiz July 20, 2013 at 6:56 am

We will focusing on importing a Gradle-based Android project into Eclipse ADT first. What you are asking for is not in our roadmap yet.

Reply

ken July 25, 2013 at 6:57 am

Is there any partially working Android Gradle support for Eclipse?
Simple stuff like “make me a project that can be build w/ gradle on the command line for releases but can still be developed using Eclipse”. Stuff like we’ve been able to do w/ Ant for a while? I mainly want to be able to include maven dependencies in Eclipse but use gradle for the command line builds.

I tried Android Studio and it doesn’t automatically create the different debug/release/free/paid/pro variants that they showed at Google I/O and was very fragile (build broke going from Android Studio 0.1.x to 0.2.x)-: and doesn’t support unit tests so IMHO, it’s not useful for day-to-day development yet :-(
So my wish list, if you guys are taking suggestions:
- doing “new Android project” should create an app that can support all the variants maybe via a checkbox to choose variants
- each variant should be able to use a different license key (and passwords should be able to be placed somewhere on the system but not in build.gradle)
- each variant should build the final APK into a different filename or directory
- each variant should be able to rename the root classpath of your app (so you can have free and paid/pro versions of an app)
- each variant can include different dependencies (e.g., one maven library used for free versions, another for paid)
- create a unit test tree that is run when building each variant; we also need common unit tests and separate ones for each variant in case there are unique features to check in e.g. a paid variant
- doing “new Android library” should build a gradle project that can upload to a local maven repo
- include an option to add an upload to OSS/Sonatype target if you’re doing an open source library and want to put it up as a public maven artifact

And make sure to have some way to work around the Gradle quirk where multiple files at the same root level in a dependency gives it fits. E.g., if you have a project that depends on junit’s jar which has LICENSE.txt and another dependency w/ the same filename, when you do a refresh, it’ll complain there’s a file conflict and stop. I’ve been using the non-Android Eclipse Gradle plugin and that’s an ugly one.

And let us specify where the downloaded maven artifacts go. The non-Android Eclipse Gradle plugin dumps them into my home directory on Windows and I’d rather put them onto a temp drive.

Thanks for listening if you’ve gotten this far :-)

Reply

Alex Ruiz July 25, 2013 at 7:35 am

Ken,

Thank you for the feedback. There is no partially-working Android-Gradle support for Eclipse yet.

Android Studio is not 1.0 yet, its status is l “I/O preview.” It is not stable and there are still bugs to fix. Here are some answers to some of the issues you mentioned:

- The only variants that are generated are “debug” and “build.” If you need more you’ll need to edit the build.gradle files by hand
- The APK for each variant is built in a separate directory, with different names
- Variants have their own set of dependencies

Support for unit testing is coming before we go 1.0. We know that editing build.gradle files by hand is ugly, we will provide dialogs where you can configure your project.

Your feedback is great. Please help us create a better product. Can you please file tickets for the things you’d like to see, using this link? We care about user feedback :-)

Thank you!

Reply

ken July 30, 2013 at 2:27 pm

Thanks, Alex. Will put in the feature requests.

I’ve been having a royally frustrating time trying to get Gradle to work w/ an Android library as well, so hopefully, you’ll test configurations like this in Eclipse and Android Studio:
library A (pure Java library…built w/ Gradle and has unit tests)
library B (Android library that depends on android support v4 and has unit tests and also depends on library A which is in Maven Central but I have the source code locally for debugging)
app that depends on library B (but I have the source code locally for debugging)

Gradle is giving me all sorts of grief about not finding the android-library plugin when trying to build library B so it can’t find any of the Android references :-P

Hope you guys get at least the Gradle stuff closer to 1.0 soon…it’s nice getting out of XML build config hell, but Gradle is currently a different sort of hell because things just don’t work right…and I find Groovy better than editing XML ;-)

Reply

ken July 30, 2013 at 3:10 pm

never mind…fixed the issue w/ the android-library plugin…started writing the build.gradle from scratch and it worked :-P

Reply

Leave a Comment

{ 1 trackback }

Previous post: