Archive for the ‘Android’ Category

Gradle surviving tricks

Sunday, November 30th, 2014

Gradle did come to stay with us. Although existing before Android Studio came, it was the Google IDE the tool that popularised it. But are we using the most of it?

After meeting some colleagues and other developers at different conferences, I have realised that the community is underusing Gradle and not exploiting all its possibilities. With this article, I want to share a few tricks I have learned that focus on improving productivity and helping to the Continuous Integration process:

Keeping multiple builds with different icons, names and package extensions

Very likely, at your organisation the Continuous Integration process will create different binaries depending on the build type (alpha, beta and release). For your team will be important to keep a copy of the different versions, so for instance you can keep the last beta version and the release version. By default, Android will keep only a package name (which substitutes each previous install) and the same icon. We can easily modify this, to allow all the different builds to co-exist together.

We can set up in the buildType a new applicationIdSuffix and a new versionNameSuffix.

For each build that we have created, we must also add under the src folder a new folder with the name of the build type. Any content within this folder will overwrite the original one when the application is being compiled in a particular build type (i.e., if we add a different icon it will be taken from here).

 

Screen Shot 2014-11-30 at 18.12.28 copy

And as we can see, all the different build types will be co-existing together.

screenshot

Using different values depending on the build type.

A very recurrent problem: we want to manually handle different values that correspond with different build types (for example, we will want to call a different URL, or track to a different Google Analytics account, depending if the application is the production one or not). By using the token buildConfigField, this task is very trivial with Gradle (and removes all the risk associated with the manual handling!)

Screen Shot 2014-11-30 at 18.00.38 2

Increasing the version code automatically.

We have already written about automatically increasing the versionCode and an extended version. The idea behind it is that every Jenkins build will take care of increasing the versionCode and therefore notifying the recipients that a new version is available. This is a really cool feature that helps us to keep a real track of all the different APKs that we are generating.

 

Handling duplicates in dex files

Did you ever see an error saying something like Multiple dex files define Lwhatever/package? It really sucks. This means you are adding twice a file that is include in one of your dependencies. If you are working with this projects this sucks twice, since you might have trouble to identify exactly which package is causing this. We can call gradlew dependencies to find in the root project where this is happening, but this is not working with subprojects (and with the current version there is no –recursive or –include-submodules flag).

There is however a small workaround. We can define this in our root build.gradle file:

which basically executes gradlew dependencies for all the subprojects. This will tell us exactly in a moment where are our dependencies, so you can eliminate them. By calling gradle listAllDependencies you will get something similar to the following paragraph:

I always have this task for all my projects. Since falling into the multiple dex error is probable, I know I can always use this task rather than checking the last changes in the project.

Your build process will increase and gets more complex as your company or organisation grows, is nothing static at all. Those tricks have helped me increased the productivity, and I have now included them by default in any project I work with.

Protected: Mobile Tech Talks 11.09.2014

Thursday, September 11th, 2014

This content is password protected. To view it please enter your password below:

How to store the Credentials securely in Android

Saturday, August 9th, 2014

Most of the applications in the Market store credentials (let it be username/password or a hash) in the application to avoid prompting the user continuously asking him for the authentication. While being a good pattern to make the application usage more fluent, it is not exempt of security risks.

Android provides different options to store data from an application. From all those, the SharedPreferences seems to be the most adequate to store credentials: it stores primitive types in an XML file, which is in principle saved in the application package folder and hidden to the user in most cases. A device that has been rooted, however, can access this folder and retrieve the SharedPreferences files, exposing the credentials if we have chosen to store them this way.

This problem also affects integrity of another applications, and allows operations such as game cheating. Srinivas proves in his blog how is possible to edit the score in Stick Cricket and submit a modified one to the server.

A rooted device always makes security harder, but we can ensure that our authentication is securely stored by combining a few techniques. Encryption is the first thing that comes in mind, but if we want to encrypt we might need to use a key, that also needs to be stored somewhere else. NDK comes here into the scene: we can store using native code the key or sequence we will use to encrypt our data. Let’s see it step by step.

First, we will write two functions to encrypt and decrypt our data. We will use AES for our purpose:

The functions are very straight-forward. Now we can use them like follows:

If we use this functionality to encrypt our data, at this point it will be encrypted but an attacker could still decompile our APK, check the key used to encrypt the data and steal our credentials. With NDK we can store this key in an .so file, making it very hard for the attacker (.so files can hardly be decompiled, and disassembling those files is far from being a comfortable option). To store and retrieve our key, we need to write the following code in our application:

And save in a file using NDK the following function:

Now we can easily retrieve our key and use it to encrypt our data.

Security is never granted and never absolute, so the solution is to always take smart design decisions and use all the available tools to prevent unauthorised usage of our applications or systems. By applying this technique you can ensure a high level of security for your application credentials!

Comprehensive guide to export your Android application to Blackberry

Tuesday, July 15th, 2014

Since the Playbook OS 2.0 was released, Blackberry provided a set of tools and procedures to export an .APK file into a Blackberry .BAR file. To put something big into small words, it is possible to repackage an .APK file into a .BAR, sign it afterwards and being able to run it on a Blackberry device. The procedure, however, turns a little bit tricky due to several reasons. The support from Blackberry does not cover a high detailed documentation, and it is far from being straightforward. In this article we will try to explain it on an easy way, using the latest tools available at this moment.

Before getting our own .APK, there are some considerations to be done.

  • Blackberry development is also Java based, so basically we will take our compile .dex files and create a virtual instance in a Blackberry machine that will run our Android application
  • Android is an OpenSource framework, but not everything in the Android SDK is OpenSource. Google Play Services, Google Maps, etc… are frameworks that have not been open-sourced from Google. We just download the libraries with a set of APIs to call the functionality, but this does not mean we have access to the code itself. Therefore there is a bunch of things from our applications that will not work in Blackberry!

This brings us to the first topic: if our application is making use of Google Maps or Google Play Services, we will need to get rid of this functionality. Google Play Services provides us typically the Location API, Analytics or the Drive API. If we have been using one of those, we will need to find an alternative (typically downloading external JARs if available, and adapting our application). For the location API, is still possible to call the old LocationManager, which is less comprehensive but it does what it needs to do.

If our application is making use of Google Maps v1, OSmdroid provides an alternative to more or less easily replace our Google Maps. The syntax is quite similar (except the part involving the markers management). For more information, you can check the OSmdroid project repository.

If our application has been using Google Maps v2, things will get more interesting. OSmdroid provides a Wrapper to be used and switch between the different maps engines. However, this wrapper does not follow the same syntax as Google Maps v2. This means you will need to invest some time in developing your application making use of this Framework if you want to make it work. In addition, the wrapper still does not cover much of the Google Maps v2 functionality. This is a decision that involves taking into account the nature of the project, requirements and some time considerations.

If your application does not use any of the private Google libraries, congratulations. You are closer to get the finest .bar file that will compile in Blackberry for the same effort as developing only its Android peer. Let’s begin to explain this.

      1. The first step is to request permission to sign your Blackberry apps. This can be done at the following website. When you are creating the .CSJ files, remember to write down the PINs! Blackberry does not provide any reset mechanism, so you will need to go over this article again if you loose them.
      2. Shortly after you will receive two files (xxxRDKxxx.csj and xxxPDBTxxx.csj at your email). Download them into your hard drive, and then execute the following command:

        This command will create a few files that you will need to use to sign up your files. Pay attention to the files author.p12, barsigner.csk and barsigner.db. Is not a bad idea to make a backup of the files inmediately. If you need to restore them in the future, you just can copy them into ~/Library/Research In Motion.
      3. Now we have our keystores and certificates. It is time to package our Blackberry application. Download the Command Line tools from the Blackberry website, uncompress them and save them into a folder of your choice (at this point is always good to remember that taking some time to add this folder to your Path will save you much more time in the future). Look at a file called Blackberry APK Packager, and open it. Something similar to the following screen will appear:Bildschirmfoto 2014-07-15 um 18.24.35
      4. You need to indicate which .APK do you want to import, and where do you want to generate a .BAR file. The Advanced Settings section allows us to set information such as author, debug mode and so on – not very relevant at the moment. If you click in Package the process will begin, and you will see if there is any unsupported package (ATTENTION! Just using classes not supported by Blackberry, such as LatLng, will make your application crash with a ClassNotFoundException). Let’s imagine you have done your homeworks, and the application has been packaged with no problems.
      5. The last step is to sign the application with the certificate, so it can run on your Blackberry device. Proceed with the following command:

        The keystore password is the one you used when you create the developer certificate, and the keystore password, and the csk password specifies the keyword you defined when you configured your computer to sign apps. The developer certificate is the author.p12 file you created before.
      6. We are a step closer. First, activate the development mode in your device (normally you will find it in “settings – security and privacy – Development Mode”). You will need to write down the IP of your device (with the USB cable connected to your computer), and the development PIN that will be prompted to write. Now, with this last information write the following in your shell:

        Changing obviously the information to your IP, package route and password you selected before.

If everything went well, your application should now be running on your Blackberry device! The process is however far from being simple. If Blackberry stakeholders are really willing to improve the platform features and make it more human-doable, they should probably focus in the signing process instead of releasing music videos. Just one of those things.

 

Increasing the performance of Gradle builds

Wednesday, April 23rd, 2014

Lately, I have been immersed into adding a bunch of new projects to our CI server. Although we have been using a distributed system to achieve a parallel build, at some point builds were taking a big amount of time. To put numbers: consider an scenario with 49 different projects (backend, frontend, mobile), in different branches (production, development) constantly building and deploying. There was an average waiting list to build projects of more than 20 minutes, with some projects taking more than 10 minutes to build and deploy. Something against the spirit of CI, really. After doing some research, I increased the performance of my builds in about one third. This is how to achieve it:

So with the problem detected, the next step was to find a solution: how to improve the performance of the builds. The first platform to improve was Android (15 from all our projects are Android based, which is around one third of the total). We are using Gradle build system and Android Studio. While is still great, is an on-going product with constant releases and has not reached its peak of performance yet . First, the important point was to identify the bottlenecks. I used the following script in our build.gradle file to detect which tasks were more problematic:

This code is relatively straight forward. For each task being executed by Gradle, will measure the time required, and when the build is finished will print the amount of time each task needed.

In order to perform a right benchmarking, I would use my computer with no extra program rather than the console, and run gradle clean assembleRelease. I run this in one of our ship projects with a quite typical structure for our company: a single project, containing 6 maven libraries and 2 local ones.

My first experiment shown nothing really surprising: I run gradle clean mergeReleaseResources, preDexRelease and dexRelease were the tasks more time consuming. Particularly:

Bildschirmfoto 2014-04-23 um 15.49.43

Pre-dexing is used for speeding up incremental builds. Pre-dexes dependencies of a module, so that they can just be merged together into the final dex file, but won won’t affect the release build (since you should be doing clean builds for release builds anyway). So we could get rid of this process during the release build:

While doing some research met two options to be used with gradlew:
–parallel executes parallel build for decoupled projects, leading to an increase in performance.
–daemon allows to execute our gradle as a daemon, speeding up the build time.

This options can be called from the console:

Or can be included in a gradle.properties file:

Combining all the points: I run again the same command and got the following:

Bildschirmfoto 2014-04-23 um 15.57.03

The increase in performance have been also successful in subsequent builds, and it is on average a 30% faster than the non-optimized version.

Which city has the most intense Android scene in Europe?

Saturday, March 8th, 2014

StackExchange Data Explorer is an open source tool to run SQL queries against public data from StackOverflow. Since StackOverflow is the biggest development forum of the world, there is surely a lot of information that companies can actually retrieve from their system in order to take some business decision (this is actually a brilliant place to apply BigData)

Moving now to different issues: I was discussing with some event organizers the possibility of bringing an Android event from the USA to Europe. Since I do live in Munich (and besides being a trendy mobile city, I think is a really cool place to organize such events), I was trying to convince them that Munich was the choice. They were resilient about it, so I needed to prove with some data that Munich would be a really nice election.

At this time, I was thinking how could I use Data Explorer and BigData to support my thesis. I remember two times I used it before, to display the most active developers from Barcelona (the city I lived before), Munich, and the two cities combined. Something similar could be a valid approach. In the previous SQL queries, I was clustering the top developers from each city based on their contribution to questions tagged with the token “android“. So I could possibly group all the developers contribution from a certain city with questions tagged with the same token. I come over with this script:

I decided to search in Germany, Spain, Holland/Netherlands, France, Italy, United Kingdom, Poland and Sweden (not that Sweden is a big country in terms of population, but I do work with a bunch of Swedish colleagues :-) ). I did some little experiments to get rid of some nomenclature errors and statistical noise (for instance, I tried also with England, thinking that some developers might registered their selves as English residents instead of UK). After some refining, I came with the following result:

Screen Shot 2014-03-08 at 9.56.08 PM

There is a major preponderance of cities from Germany and the UK, with only 5 cities from different countries in the first 20 cities against 9 british cities and 6 Germans. Is not my purpose here to give a full sociologic analysis (i.e., many cities are from UK since StackOverflow is an English based community and there are other local communities in different countries – although 80% of the Internet documentation is English based), but to give a rough approach.

So, now we have a bunch of cities with some numbers. We still need to go through a normalization process (i.e., cities with more population will always have more UpVotes than cities with less population). Since StackOverflow does not provide statistics for cities population (is not either their task) I will correlate this values manually. Thus, I will assign to each city the factor that determines the relationship between UpVotes and population. To get the population value, I did use Wikipedia.

  1. London: 1.783.457 / 8.174.000 = 0.21
  2. Reading: 701.683 / 145.700 = 4.815
  3. Berlin: 570.379 / 3.502.000 = 0.16
  4. Paris: 381.560 / 2.234.105 =0.17
  5. Amsterdam: 335.999 / 779.808 = 0.43
  6. Cambridge: 328.603 / 123.900 = 2.65
  7. Munich: 252.516 / 1.378.000 = 0.18
  8. Frankfurt: 139.674 / 691.518 = 0.20
  9. Manchester: 126.113 / 510.700 = 0.24
  10. Lyon: 124.057 / 474.946 = 0.26
  11. Warsaw: 116.670 / 1.717.000 = 0.06
  12. Oxford: 100.504 / 150.200 = 0.66
  13. Hamburg: 979.30/ 1.799.000 = 0.05
  14. Madrid: 96.906 / 3.234.000 = 0.02
  15. Karlsruhe: 95.848 /297.488 = 0.32
  16. Ulm: 92.675/ 123.672 = 0.74
  17. Edinburgh: 84.748 / 495.370 = 0.17
  18. Brighton: 81.612 / 155.919 = 0.52
  19. Ulverston: 79.056 / 11.524 = 6.86
  20. Barcelona: 78.130 / 162.1000 = 0.04

And if we now sort it by the coefficient:

  1. Ulverston (UK): 6.86
  2. Reading (UK): 4.815
  3. Cambridge (UK):  2.65
  4. Ulm (Germany): 0.74
  5. Oxford (UK): 0.66
  6. Brighton (UK): 0.52
  7. Amsterdam (Netherlands): 0.43
  8. Karlsruhe (Germany): 0.32
  9. Lyon (France): 0.26
  10. Manchester (UK): 0.24
  11. London (UK): 0.21
  12. Frankfurt (Germany): 0.20
  13. Munich (Germany): 0.18
  14. Paris (France): 0.17
  15. Edinburgh (UK): 0.17
  16. Berlin (Germany): 0.16
  17. Warsaw (Poland): 0.06
  18. Hambug (Germany): 0.05
  19. Barcelona (Spain): 0.04
  20. Madrid (Spain): 0.02

There is some very interesting information in this graph:

  • The result of UK cities are brilliant. Reading, Cambridge and Oxford, famous for their universities, are all in the top 5
  • Ulverston, a tiny city of North West England, scores first in the ranking. It would be interesting to determine the reasons, since there is no known university or industry in the town. Probably a top user from StackOverflow explains it, but it can be discarded as statistical noise.
  • For most of the cities, a value between 0 and 1 seems the norm.
  • Big capitals are generally in the mid part of the table, except Amsterdam

After this sampling Munich does not score that bad (although better in absolute terms). There are, however, a bunch of other different reasons to choose a place for a certain event (proximity to other places, infrastructures, communication, hosting prices, global number of possible attendants, etc). But after this little experiment, I can only suggest to organizers to move to the city of Ulverston (even if I still think that Munich offers a great beer).

Follow me on Twitter @eenriquelopez !

Automatically increasing versionCode with Gradle – Followup

Wednesday, February 12th, 2014

This article is a followup to the original that can be found here.

As a normal use case the original article provides a good way not to worry about increasing the version number. The only drawback is that the number is really increased every time you compile the source code. This results in jumping versionCodes in the commit history. For instance lets say you checked out the App with the versionCode set to 3. After compiling the application for several times, until you deem it stable, you might check it in with a versionCode of 15. It would be more desirable to increase it just by one.

To achieve this we are going to base our new implementation on the committed state of the AndroidManifest.xml (here for the DriveNow project).

To incorporate this into the normal build process, we can basically use the same methodology we used in the original version. If we want to go a step further and don’t increase the version if we are running on our integration server, we can also alter the task definition a little bit.

Running the build now with  ./gradlew assemble -D buildConfig.keepVersionCode=true will build the project without increasing the versionCode.

Automatically increasing versionCode with Gradle

Wednesday, January 29th, 2014

Continuous Integration means, above all, automatization. The user should not be in charge of the distribution or deployment process. Everything should be scripted!

While deploying new versions in Android, one of the common tasks is to increase the versionCode to identify a particular build. Using the new Gradle system, this can also be automatized.

In our defaultConfig, we will need to specify that the versionCode must be read from the newly added function:

Testing Asynchronous Tasks on Android

Tuesday, January 28th, 2014

Recently, at Sixt we have been migrating our development environment from Eclipse to Android Studio. This has mean we have also moved to the new build system, Gradle, and applying TDD and CI to our software development process. This is not the place to discuss the benefits of applying CI to a software development plan, but to talk about a problem arising when testing tasks running on different threads than the UI in Android.

 

A test in Android is (broad definition) an extension of a JUnit Suitcase. They do include setUp() and tearDown() for initialization/closing the tests, and infers using reflection the different test methods (starting with JUnit 4 we can use annotations to specify the priority and execution of all the tests). A typical test structure will look like:

This is a very obvious instance: in a practical case we would like to test things such as HTTP requests, SQL storage, etc. In Sixt we follow a Manager/Model approach: each Model contains the representation of an Entity (a Car, a User…) and each Manager groups a set of functionality using different models (for example, our LoginManager might require of models Users to interact with them). Most our managers perform HTTP  requests intensively in order to retrieve data from our backend. As an example, we would perform the login of a user using the following code:

 

When it comes to apply this to our own test suitcase, we just make the process fail() when the result does not work as we were expecting. We can see why in the method onFailure() we call to fail().

However, even if I was using a wrong username the test was still passing. Wondering around, seems that the test executed the code sequentially, and did not wait until the result of the callbacks was back. This is certainly a bad approach, since a modern application do intense usage of asynchronous tasks and callback methods to retrieve data from a backend!. Tried applying the @UiThreadTest bust still didn’t work.

I found the following working method. I simply use CountDownLatch signal objects to implement the wait-notify (you can use synchronized(lock){… lock.notify();}, however this results in ugly code) mechanism. The previous code will look like follows:

Follow me on Twitter @eenriquelopez !