Gradle surviving tricks

Gradle did come to stay with us. Although existing before Android Studio came, it was the Google IDE the tool that popularised it. But are we using the most of it?

After meeting some colleagues and other developers at different conferences, I have realised that the community is underusing Gradle and not exploiting all its possibilities. With this article, I want to share a few tricks I have learned that focus on improving productivity and helping to the Continuous Integration process:

Keeping multiple builds with different icons, names and package extensions

Very likely, at your organisation the Continuous Integration process will create different binaries depending on the build type (alpha, beta and release). For your team will be important to keep a copy of the different versions, so for instance you can keep the last beta version and the release version. By default, Android will keep only a package name (which substitutes each previous install) and the same icon. We can easily modify this, to allow all the different builds to co-exist together.

We can set up in the buildType a new applicationIdSuffix and a new versionNameSuffix.

For each build that we have created, we must also add under the src folder a new folder with the name of the build type. Any content within this folder will overwrite the original one when the application is being compiled in a particular build type (i.e., if we add a different icon it will be taken from here).


Screen Shot 2014-11-30 at 18.12.28 copy

And as we can see, all the different build types will be co-existing together.


Using different values depending on the build type.

A very recurrent problem: we want to manually handle different values that correspond with different build types (for example, we will want to call a different URL, or track to a different Google Analytics account, depending if the application is the production one or not). By using the token buildConfigField, this task is very trivial with Gradle (and removes all the risk associated with the manual handling!)

Screen Shot 2014-11-30 at 18.00.38 2

Increasing the version code automatically.

We have already written about automatically increasing the versionCode and an extended version. The idea behind it is that every Jenkins build will take care of increasing the versionCode and therefore notifying the recipients that a new version is available. This is a really cool feature that helps us to keep a real track of all the different APKs that we are generating.


Handling duplicates in dex files

Did you ever see an error saying something like Multiple dex files define Lwhatever/package? It really sucks. This means you are adding twice a file that is include in one of your dependencies. If you are working with this projects this sucks twice, since you might have trouble to identify exactly which package is causing this. We can call gradlew dependencies to find in the root project where this is happening, but this is not working with subprojects (and with the current version there is no –recursive or –include-submodules flag).

There is however a small workaround. We can define this in our root build.gradle file:

which basically executes gradlew dependencies for all the subprojects. This will tell us exactly in a moment where are our dependencies, so you can eliminate them. By calling gradle listAllDependencies you will get something similar to the following paragraph:

I always have this task for all my projects. Since falling into the multiple dex error is probable, I know I can always use this task rather than checking the last changes in the project.

Your build process will increase and gets more complex as your company or organisation grows, is nothing static at all. Those tricks have helped me increased the productivity, and I have now included them by default in any project I work with.

Prototyping Web Services

An incorrect development cycle leads into inefficiency and high costs. We know what this means when we work developing mobile technologies, since there is an strict need to coordinate with more teams that deliver services to our applications.

We have came with the idea of mocking and prototyping our Backend services to speed up the development, and work parallelly with the Backend Team without creating a bottleneck until the services are delivered. For this purpose we have been using Apiary, which provides a great service but has two big limitations:

1.- Is not possible to test services that update an object in the server, such as PATCH or POST. We only receive the Mockups.
2.- We cannot filter requests based on different parameters (i.e., the content being sent is always static).

To overcome those limitations we will use Ruby on Rails (RoR). This is a web application framework written in Ruby, and is awesome. It allows you to create and deploy very easily web applications. I find it particularly cool when it comes to prototyping, since creating a web project takes the same effort as just writing the mockups in Apiary! It has many other cool features, such as autoa

We will introduce it with a very short example from a Coursera course. Let’s gonna create a basic blog application where we can store posts, and comments associated to those posts.

NOTE: RoR requires a set up on your computer that I will not cover in this article, but you can easily find instructions on the Internet.

The first step is to create the application with Ruby:

This will create a folder with the basic structure for the application.

Screen Shot 2014-09-22 at 2.00.58 PM

Our database will have two models: comments and posts. We will use scaffolds to create them:

Those two lines add the entities to our RoR application. We still need to add an association to them (a post can contain several comments). Let’s check in the folder of the application /app/models. There are two files, one for its entity. We will specify here the association between the models.

We will specify also that, if a post gets removed, the comments associated must also be removed.

Finally, with the following line we will transport those changes into our database schema.

And launch the application.

By default we can see it in http://localhost:3000/.Wwe can add posts or comments to the URL to display the particular entity. With only a few clicks, we have been able to construct a fully working application, with all the HTTP methods associated to them. If we type in the console rake routes all the routes and Entry Points of the application are displayed.

Screen Shot 2014-09-22 at 2.26.58 PM

Protected: Mobile Tech Talks 11.09.2014

This content is password protected. To view it please enter your password below:

How to store the Credentials securely in Android

Most of the applications in the Market store credentials (let it be username/password or a hash) in the application to avoid prompting the user continuously asking him for the authentication. While being a good pattern to make the application usage more fluent, it is not exempt of security risks.

Android provides different options to store data from an application. From all those, the SharedPreferences seems to be the most adequate to store credentials: it stores primitive types in an XML file, which is in principle saved in the application package folder and hidden to the user in most cases. A device that has been rooted, however, can access this folder and retrieve the SharedPreferences files, exposing the credentials if we have chosen to store them this way.

This problem also affects integrity of another applications, and allows operations such as game cheating. Srinivas proves in his blog how is possible to edit the score in Stick Cricket and submit a modified one to the server.

A rooted device always makes security harder, but we can ensure that our authentication is securely stored by combining a few techniques. Encryption is the first thing that comes in mind, but if we want to encrypt we might need to use a key, that also needs to be stored somewhere else. NDK comes here into the scene: we can store using native code the key or sequence we will use to encrypt our data. Let’s see it step by step.

First, we will write two functions to encrypt and decrypt our data. We will use AES for our purpose:

The functions are very straight-forward. Now we can use them like follows:

If we use this functionality to encrypt our data, at this point it will be encrypted but an attacker could still decompile our APK, check the key used to encrypt the data and steal our credentials. With NDK we can store this key in an .so file, making it very hard for the attacker (.so files can hardly be decompiled, and disassembling those files is far from being a comfortable option). To store and retrieve our key, we need to write the following code in our application:

And save in a file using NDK the following function:

Now we can easily retrieve our key and use it to encrypt our data.

Security is never granted and never absolute, so the solution is to always take smart design decisions and use all the available tools to prevent unauthorised usage of our applications or systems. By applying this technique you can ensure a high level of security for your application credentials!

Comprehensive guide to export your Android application to Blackberry

Since the Playbook OS 2.0 was released, Blackberry provided a set of tools and procedures to export an .APK file into a Blackberry .BAR file. To put something big into small words, it is possible to repackage an .APK file into a .BAR, sign it afterwards and being able to run it on a Blackberry device. The procedure, however, turns a little bit tricky due to several reasons. The support from Blackberry does not cover a high detailed documentation, and it is far from being straightforward. In this article we will try to explain it on an easy way, using the latest tools available at this moment.

Before getting our own .APK, there are some considerations to be done.

  • Blackberry development is also Java based, so basically we will take our compile .dex files and create a virtual instance in a Blackberry machine that will run our Android application
  • Android is an OpenSource framework, but not everything in the Android SDK is OpenSource. Google Play Services, Google Maps, etc… are frameworks that have not been open-sourced from Google. We just download the libraries with a set of APIs to call the functionality, but this does not mean we have access to the code itself. Therefore there is a bunch of things from our applications that will not work in Blackberry!

This brings us to the first topic: if our application is making use of Google Maps or Google Play Services, we will need to get rid of this functionality. Google Play Services provides us typically the Location API, Analytics or the Drive API. If we have been using one of those, we will need to find an alternative (typically downloading external JARs if available, and adapting our application). For the location API, is still possible to call the old LocationManager, which is less comprehensive but it does what it needs to do.

If our application is making use of Google Maps v1, OSmdroid provides an alternative to more or less easily replace our Google Maps. The syntax is quite similar (except the part involving the markers management). For more information, you can check the OSmdroid project repository.

If our application has been using Google Maps v2, things will get more interesting. OSmdroid provides a Wrapper to be used and switch between the different maps engines. However, this wrapper does not follow the same syntax as Google Maps v2. This means you will need to invest some time in developing your application making use of this Framework if you want to make it work. In addition, the wrapper still does not cover much of the Google Maps v2 functionality. This is a decision that involves taking into account the nature of the project, requirements and some time considerations.

If your application does not use any of the private Google libraries, congratulations. You are closer to get the finest .bar file that will compile in Blackberry for the same effort as developing only its Android peer. Let’s begin to explain this.

      1. The first step is to request permission to sign your Blackberry apps. This can be done at the following website. When you are creating the .CSJ files, remember to write down the PINs! Blackberry does not provide any reset mechanism, so you will need to go over this article again if you loose them.
      2. Shortly after you will receive two files (xxxRDKxxx.csj and xxxPDBTxxx.csj at your email). Download them into your hard drive, and then execute the following command:

        This command will create a few files that you will need to use to sign up your files. Pay attention to the files author.p12, barsigner.csk and barsigner.db. Is not a bad idea to make a backup of the files inmediately. If you need to restore them in the future, you just can copy them into ~/Library/Research In Motion.
      3. Now we have our keystores and certificates. It is time to package our Blackberry application. Download the Command Line tools from the Blackberry website, uncompress them and save them into a folder of your choice (at this point is always good to remember that taking some time to add this folder to your Path will save you much more time in the future). Look at a file called Blackberry APK Packager, and open it. Something similar to the following screen will appear:Bildschirmfoto 2014-07-15 um 18.24.35
      4. You need to indicate which .APK do you want to import, and where do you want to generate a .BAR file. The Advanced Settings section allows us to set information such as author, debug mode and so on – not very relevant at the moment. If you click in Package the process will begin, and you will see if there is any unsupported package (ATTENTION! Just using classes not supported by Blackberry, such as LatLng, will make your application crash with a ClassNotFoundException). Let’s imagine you have done your homeworks, and the application has been packaged with no problems.
      5. The last step is to sign the application with the certificate, so it can run on your Blackberry device. Proceed with the following command:

        The keystore password is the one you used when you create the developer certificate, and the keystore password, and the csk password specifies the keyword you defined when you configured your computer to sign apps. The developer certificate is the author.p12 file you created before.
      6. We are a step closer. First, activate the development mode in your device (normally you will find it in “settings – security and privacy – Development Mode”). You will need to write down the IP of your device (with the USB cable connected to your computer), and the development PIN that will be prompted to write. Now, with this last information write the following in your shell:

        Changing obviously the information to your IP, package route and password you selected before.

If everything went well, your application should now be running on your Blackberry device! The process is however far from being simple. If Blackberry stakeholders are really willing to improve the platform features and make it more human-doable, they should probably focus in the signing process instead of releasing music videos. Just one of those things.


Increasing the performance of Gradle builds

Lately, I have been immersed into adding a bunch of new projects to our CI server. Although we have been using a distributed system to achieve a parallel build, at some point builds were taking a big amount of time. To put numbers: consider an scenario with 49 different projects (backend, frontend, mobile), in different branches (production, development) constantly building and deploying. There was an average waiting list to build projects of more than 20 minutes, with some projects taking more than 10 minutes to build and deploy. Something against the spirit of CI, really. After doing some research, I increased the performance of my builds in about one third. This is how to achieve it:

So with the problem detected, the next step was to find a solution: how to improve the performance of the builds. The first platform to improve was Android (15 from all our projects are Android based, which is around one third of the total). We are using Gradle build system and Android Studio. While is still great, is an on-going product with constant releases and has not reached its peak of performance yet . First, the important point was to identify the bottlenecks. I used the following script in our build.gradle file to detect which tasks were more problematic:

This code is relatively straight forward. For each task being executed by Gradle, will measure the time required, and when the build is finished will print the amount of time each task needed.

In order to perform a right benchmarking, I would use my computer with no extra program rather than the console, and run gradle clean assembleRelease. I run this in one of our ship projects with a quite typical structure for our company: a single project, containing 6 maven libraries and 2 local ones.

My first experiment shown nothing really surprising: I run gradle clean mergeReleaseResources, preDexRelease and dexRelease were the tasks more time consuming. Particularly:

Bildschirmfoto 2014-04-23 um 15.49.43

Pre-dexing is used for speeding up incremental builds. Pre-dexes dependencies of a module, so that they can just be merged together into the final dex file, but won won’t affect the release build (since you should be doing clean builds for release builds anyway). So we could get rid of this process during the release build:

While doing some research met two options to be used with gradlew:
–parallel executes parallel build for decoupled projects, leading to an increase in performance.
–daemon allows to execute our gradle as a daemon, speeding up the build time.

This options can be called from the console:

Or can be included in a file:

Combining all the points: I run again the same command and got the following:

Bildschirmfoto 2014-04-23 um 15.57.03

The increase in performance have been also successful in subsequent builds, and it is on average a 30% faster than the non-optimized version.

Export XML files to CSV (or Android I18N files to iPhone I18N files)

Android and iOS have two different peculiarities when it comes to I18N: they present different file formats. Google platform uses XML, whereas Cupertino’s uses CSV files. Lately we came to a problem to unify some of out I18N resources. In order to do that, we wanted to have a unified format to easily compare whether the strings where correctly modified or not.

This Ruby script was designed to export Android strings into a .CSV file (and therefore, really easy to script if you need to compare it with its Apple pair). Remember you need to install Nokogiri to make it work.

And call it with ruby scriptname.rb inputfile.xml outputfile.csv

Which city has the most intense Android scene in Europe?

StackExchange Data Explorer is an open source tool to run SQL queries against public data from StackOverflow. Since StackOverflow is the biggest development forum of the world, there is surely a lot of information that companies can actually retrieve from their system in order to take some business decision (this is actually a brilliant place to apply BigData)

Moving now to different issues: I was discussing with some event organizers the possibility of bringing an Android event from the USA to Europe. Since I do live in Munich (and besides being a trendy mobile city, I think is a really cool place to organize such events), I was trying to convince them that Munich was the choice. They were resilient about it, so I needed to prove with some data that Munich would be a really nice election.

At this time, I was thinking how could I use Data Explorer and BigData to support my thesis. I remember two times I used it before, to display the most active developers from Barcelona (the city I lived before), Munich, and the two cities combined. Something similar could be a valid approach. In the previous SQL queries, I was clustering the top developers from each city based on their contribution to questions tagged with the token “android“. So I could possibly group all the developers contribution from a certain city with questions tagged with the same token. I come over with this script:

I decided to search in Germany, Spain, Holland/Netherlands, France, Italy, United Kingdom, Poland and Sweden (not that Sweden is a big country in terms of population, but I do work with a bunch of Swedish colleagues :-) ). I did some little experiments to get rid of some nomenclature errors and statistical noise (for instance, I tried also with England, thinking that some developers might registered their selves as English residents instead of UK). After some refining, I came with the following result:

Screen Shot 2014-03-08 at 9.56.08 PM

There is a major preponderance of cities from Germany and the UK, with only 5 cities from different countries in the first 20 cities against 9 british cities and 6 Germans. Is not my purpose here to give a full sociologic analysis (i.e., many cities are from UK since StackOverflow is an English based community and there are other local communities in different countries – although 80% of the Internet documentation is English based), but to give a rough approach.

So, now we have a bunch of cities with some numbers. We still need to go through a normalization process (i.e., cities with more population will always have more UpVotes than cities with less population). Since StackOverflow does not provide statistics for cities population (is not either their task) I will correlate this values manually. Thus, I will assign to each city the factor that determines the relationship between UpVotes and population. To get the population value, I did use Wikipedia.

  1. London: 1.783.457 / 8.174.000 = 0.21
  2. Reading: 701.683 / 145.700 = 4.815
  3. Berlin: 570.379 / 3.502.000 = 0.16
  4. Paris: 381.560 / 2.234.105 =0.17
  5. Amsterdam: 335.999 / 779.808 = 0.43
  6. Cambridge: 328.603 / 123.900 = 2.65
  7. Munich: 252.516 / 1.378.000 = 0.18
  8. Frankfurt: 139.674 / 691.518 = 0.20
  9. Manchester: 126.113 / 510.700 = 0.24
  10. Lyon: 124.057 / 474.946 = 0.26
  11. Warsaw: 116.670 / 1.717.000 = 0.06
  12. Oxford: 100.504 / 150.200 = 0.66
  13. Hamburg: 979.30/ 1.799.000 = 0.05
  14. Madrid: 96.906 / 3.234.000 = 0.02
  15. Karlsruhe: 95.848 /297.488 = 0.32
  16. Ulm: 92.675/ 123.672 = 0.74
  17. Edinburgh: 84.748 / 495.370 = 0.17
  18. Brighton: 81.612 / 155.919 = 0.52
  19. Ulverston: 79.056 / 11.524 = 6.86
  20. Barcelona: 78.130 / 162.1000 = 0.04

And if we now sort it by the coefficient:

  1. Ulverston (UK): 6.86
  2. Reading (UK): 4.815
  3. Cambridge (UK):  2.65
  4. Ulm (Germany): 0.74
  5. Oxford (UK): 0.66
  6. Brighton (UK): 0.52
  7. Amsterdam (Netherlands): 0.43
  8. Karlsruhe (Germany): 0.32
  9. Lyon (France): 0.26
  10. Manchester (UK): 0.24
  11. London (UK): 0.21
  12. Frankfurt (Germany): 0.20
  13. Munich (Germany): 0.18
  14. Paris (France): 0.17
  15. Edinburgh (UK): 0.17
  16. Berlin (Germany): 0.16
  17. Warsaw (Poland): 0.06
  18. Hambug (Germany): 0.05
  19. Barcelona (Spain): 0.04
  20. Madrid (Spain): 0.02

There is some very interesting information in this graph:

  • The result of UK cities are brilliant. Reading, Cambridge and Oxford, famous for their universities, are all in the top 5
  • Ulverston, a tiny city of North West England, scores first in the ranking. It would be interesting to determine the reasons, since there is no known university or industry in the town. Probably a top user from StackOverflow explains it, but it can be discarded as statistical noise.
  • For most of the cities, a value between 0 and 1 seems the norm.
  • Big capitals are generally in the mid part of the table, except Amsterdam

After this sampling Munich does not score that bad (although better in absolute terms). There are, however, a bunch of other different reasons to choose a place for a certain event (proximity to other places, infrastructures, communication, hosting prices, global number of possible attendants, etc). But after this little experiment, I can only suggest to organizers to move to the city of Ulverston (even if I still think that Munich offers a great beer).

Follow me on Twitter @eenriquelopez !

Automatically increasing versionCode with Gradle – Followup

This article is a followup to the original that can be found here.

As a normal use case the original article provides a good way not to worry about increasing the version number. The only drawback is that the number is really increased every time you compile the source code. This results in jumping versionCodes in the commit history. For instance lets say you checked out the App with the versionCode set to 3. After compiling the application for several times, until you deem it stable, you might check it in with a versionCode of 15. It would be more desirable to increase it just by one.

To achieve this we are going to base our new implementation on the committed state of the AndroidManifest.xml (here for the DriveNow project).

To incorporate this into the normal build process, we can basically use the same methodology we used in the original version. If we want to go a step further and don’t increase the version if we are running on our integration server, we can also alter the task definition a little bit.

Running the build now with  ./gradlew assemble -D buildConfig.keepVersionCode=true will build the project without increasing the versionCode.

Colour your git

You want autocompletion, colours and your current branch in your terminal? You can find hundreds of tutorials on the net, I will just tell you what to do on MacOS:

Add these lines to your .bash_profile

This will enable autocompletion for git commands and branch names and also show your current branch. Note that in the last line, you can change colours if you’d like to be different.

You may need these two files in /usr/local/git/contrib/completion.

To enable fancy colours in your git command output, type:

or just some of these if you don’t want everything coloured:

Enjoy your all-new Terminal.