Some context first
We’ve recently taken over an existing project which suffered from some significant issues with the build and deployment processes: builds were tied to a specific machine and developer account; credentials were manually managed; and release processes were handled manually and tracked in spreadsheets.
We’ve updated this to a build and deployment process managed through merge requests, which automatically creates builds targeting different environments - enabling comprehensive testing and rapid app updates.
We’re now in the post React Native upgrade era. Our philosophy for this exercise was “first make it work, then make it better”.
We started by automating as much as possible, and then improving what needs to be done. We had 2 short-term objectives:
- To have builds for each environment - previously there were several builds for each environment, and it was quite an intricate process to see which build was being rolled out.
- We wanted these builds to mirror our branching model. Merges to the development branch should result in a build connected to the test environment; the release branch should connect to staging: and the master should connect to the production environment.
- Following this logic results in every build being connected to the appropriate environment. From there, it becomes obvious when installing the app which environment is being targeted.
- We also wanted to make these apps available to external users through TestFlight, and to indicate the targeted environment in the app icon. This removed the need to track releases with a spreadsheet.
- To have merge request builds for QA, enabling QAs to easily-install builds on their test devices directly from the merge request.
For the sake of efficiency, we wanted our solution to be familiar, simple to use and at hand.
Since the code was already in a GitLab repository, we started by exploring Gitlab Pipelines. That dream was short-lived, because the runners in GitLab Pipelines are Linux-based and aren’t able to build iOS applications.
We then started looking for alternatives:
- Creating custom runners on GitLab. Here, there were two options:
- Configuring a physical Mac machine as a GitLab runner
- Using a cloud-based Mac service (MacStadium or similar)
- A fully managed cloud service, involving some one-off configuration. We looked at two options:
- CircleCI, which doesn’t offer GitLab integration
Bitrise is a fun and useful tool - so why did we choose it?
- It integrates with GitLab
- It offers MacOS Virtual Machines
- The “concurrencies” features allows parallel running of several builds
- Workflows are formed with build steps, and then formed into pipelines
Our first question was how we wanted to build iOS and Android apps - in parallel or consecutive builds?
Parallel vs consecutive builds
Running the Android and iOS builds simultaneously using the two concurrencies offered by BitRise is the faster of the two approaches, with the overall duration lasting the length of the longest build.
The consecutive approach, by comparison, provides the ability to run two different builds - for example, two branches or two merge requests. This is a lot slower, as the overall duration is the sum of the time taken for both builds.
There are also some other differences - parallel builds clone the repository and fetch dependencies at the start of each parallel run, whereas the consecutive approach does this only once for both runs.
Build logs and build artefacts are also handled differently - the parallel approach consolidates these into a single location, whereas consecutive builds result in multiple versions.
Having considered all of the above, we decided to go with the parallel approach.
This is what the results looks like:
The build is triggered by a push or merge request. The first script, built by us, determines the type of build we want to do (testing, staging or production). A parallel build is then started for iOS and the main thread continues the build for Android.
Once all the steps are executed, the main thread waits for the iOS build to finish and then reports the status back to Gitlab. This is the main reason why the Android main thread must wait for the iOS build.
Bitrise also helped us a lot by offering many out-of-the-box steps. With just a quick glance, we can see what actions are happening and which steps are needed for the build to be successful.
As helpful a tool as Bitrise proved to be, the process still wasn’t as smooth as we had hoped.
We encountered some issues along the way:
- Problems controlling whether secrets contained in the environment variables are exposed to merge requests or not.
- Issues with configuring the code signing process, a mandatory step for every release build for both Android and iOS.
- Handling two-factor authentication to retrieve certificates from Apple’s servers. This was solved by creating a session using other methods to retrieve the assets, and offering those to the build pipeline
- Managing Apple’s application-specific passwords, which are required to upload the app to the App Store.
- Improving build speed with cached dependencies
After a lot of trial and error we eventually started seeing consistent success.
This example shows an Android build - it’s the primary workflow for a merge request. At a glance it’s possible to see that this is merge request 164 which targets development.
Here, a build is started directly from development in response to a push or merge request.
Builds can also be started on demand to make a custom configuration, for example.
One of the most important steps in the build process is Fastlane.
When we took over the project, it was already running with Fastlane, saving time when it came to build and release. We also like the fact that it shows how many hours we save!
Fastlane started out initially as a tool for iOS developers to help with the complexity of managing certificates, signing and pushing apps to the App Store. It is open source, and automates the entire process from build to deployment, as well as handling screenshots and release notes.
Based on Ruby scripts, it runs from the command line without any interface interaction required. It’s also decoupled from the continuous integration and continuous deployment systems.
Each release track or build track is called a “lane”. We need a feature lane to build for QA, a staging lane for staging builds, and a production lane for production builds.
How does Fastlane save time?
Making a test build involved a number of manual steps:
- Registering test devices on Apple’s portal.
- Generating development, provisioning and deployment certificates
- Generating provisioning profiles that link devices with specific certificates to allow install of the builds on that device.
- Downloading everything on the local development machine.
- Understanding the dependencies, then building the source code and sharing the results.
With Fastlane, everything is more efficient. Configuration steps are defined in “lanes” as simple Ruby scripts. The match function makes sure you always have the necessary certificates and the provision profiles. If they don’t exist, it creates them; if they exist, it downloads them for you; if you change devices, it re-generates the provision profiles in the background.
Cocoapods is used to install dependencies, then Fastlane does the app build. By doing all these actions, it saves developers a lot of time.
What did we do with it?
- We automated the process and made some improvements
- Information about the environment and release notes were added to TestFlight, so that it was possible to identify which version of the app targeted which environment
- Through tweaking our git processes, the targeted environment was included in the commit message. For developers, the relevant information is now right there in the code - for stakeholders, the information is embedded in the app.
- TestFlight and Google Play were set up to support multiple app builds and environments
- Fastlane’s capabilities were extended to decorate the app icons with environment and build number
Release Process as Code
With Fastlane, we’ve built our release process in code. As it goes for pretty much everything, this too has advantages and disadvantages.
- The entire release process is secured under version control in the git repository. Everyone is able to contribute to it in a managed way.
- Fastlane is easy to combine with continuous integration systems because it is a command line tool.
- It can also be run locally for testing, with fast feedback loops. Local builds take around 10 mins, 20% of the time needed on Bitrise. Debugging is significantly faster.
- There’s support for both iOS and Android and Fastlane works well with React Native.
- Documentation for lanes is automatically generated, so if someone from the team wants to see how to use this tool, the information is right there in the documentation.
- There’s a learning curve - but this naturally occurs when trying to use any new tool for continuous deployment and integration automation.
Basically, from our point of view, Fastlane + Bitrise = Love 💜.
Fastlane helps us to manage the release process in code, while Bitrise helps us take our benchmarking system, identify what lanes to start and what environments to use for that lane. Putting these two together has proven to be the optimum solution for us for now.
But we’re not finished yet - there’s still some steps that will further improve our processes:
- Lane modularity: one lane contains 5 steps, with the final step being to upload the app to the Play Store or TestFlight. We’ll split these steps into smaller pieces to be able to take one piece, have a build or, if we already have the build, upload it to Play Store.
- Build time: it’s currently taking around 1 hour, and there’s still room for improvement there.
- Many third party integrations such as Slack, Jira etc. can be done in Fastlane or directly in Bitrise - but some can’t be resolved as swiftly and must be modified in the application.
- After implementing these processes and seeing the pros and cons first-hand, we concluded that having different builds for test and staging might offer more disadvantages in the long run. In the future we’ll have a single build of the application from which the Testing engineer can dynamically select the environment and targeted API when starting the app.
- We’ll further-automate as much of the release process as possible - release notes, for example, or automatically pushing it to the clients who are testing.
- Introducing end-to-end automated tests in the app. These will be added to the process so we’ll learn about any issues before the build for a merge request finishes - meaning we’ll be aware of problems without human intervention.
What we’d extract and reuse:
1. The Fastlane configuration
When you have an iOS app which builds successfully applying this configuration, whichever lane you’d define will probably be successful as well, as long as the application has already been properly configured. The same applies to Android. If gradle is correctly configured, the app builds successfully, you have all the necessary credentials running the Fastlane lane and the app will get in the Play Store. The problem is that there are still several manual steps that need to be automated to be able to make configurations in Store.
2. There are still some steps in Bitrise which could be extracted as .yml code, although this would tie the project to Bitrise. An improvement point we’d like to tackle in the future is to migrate what we have in Bitrise to Fastlane, taking advantage of the modularization that has already taken place. The decisions we made when first starting out with this process made sense at that time, but as our understanding of both Bitrise and Fastlane has deepened, new solutions have emerged.
It all comes down to finding solutions that work for our particular context, taking all the steps that we need, transposing the process into automated steps and, then extracting everything in our repository.