Mobile Test Automation Explained

What to automate on mobile

The test pyramid applies to mobile as to any product (the guide on manual vs automated testing covers why): most checks belong below the UI, where they are fast and stable. On Android, local unit tests run on the development machine’s JVM in seconds, while instrumented tests run on a device or emulator; on iOS, XCTest runs unit tests and UI tests from the same Xcode project. Business rules, validation and calculations belong in unit tests and in API tests against the back end. UI tests are for the journeys that must never break — sign-in, onboarding, search, checkout, the app’s core feature — and for the wiring between screens.

• Unit tests — view models, formatters, validation, local database queries; run on every commit.
• Instrumented or integration tests — code that needs the platform: storage, the local database, background work; run on emulators or simulators.
• API tests — the back end the app calls, including error responses and old-client compatibility.
• UI tests — a small set of critical journeys per platform, run in CI on emulators and on real devices before release.
• Manual and exploratory testing — gestures, real interruptions, look and feel, real-device behaviour (see the section on what stays manual).

Espresso, XCUITest and Appium compared

There is no universal winner. If developers own the tests and the platforms are built by separate teams, the native frameworks fit best: fastest feedback, least flakiness, same language as the app. If a separate automation team covers both platforms with shared flows, or tests must run against release-like builds without source access, Appium’s single codebase and language choice matter more. Many teams combine them: native frameworks for screen-level tests in the developer pipeline, and a small cross-platform Appium suite for end-to-end journeys.

• Espresso — Android only; tests in Kotlin or Java; runs inside the app’s process as an instrumentation test. It synchronises automatically with the app’s UI thread, so it acts only when the UI is idle; background work it cannot see is registered through idling resources. Fast and stable; usually written by Android developers alongside the code. It cannot reach system UI outside the app — UI Automator covers that.
• XCUITest — iOS only; tests in Swift or Objective-C, written in Xcode with XCTest (Apple now documents the UI testing API as XCUIAutomation). The test runner is a separate process that drives the app through the accessibility layer, as a user would. It waits for the app to settle before interacting and can handle system alerts. Usually written by iOS developers or automation engineers who work in Xcode.
• Appium — Android, iOS and other platforms; tests in any language with a WebDriver client (Java, Python, JavaScript, C#, Ruby…). A client sends W3C WebDriver commands to the Appium server, which uses a platform driver — for example the UiAutomator2 driver on Android and the XCUITest driver on iOS — to act on the device. It treats the app as a black box, can switch into WebView contexts of hybrid apps, and can automate mobile browsers. Each step crosses more layers, so it is slower than the native frameworks.

Espresso: in-process and synchronised

An Espresso test finds a view with a matcher, performs an action on it, and checks an assertion. Before every action and assertion Espresso waits until the UI thread’s message queue is idle, which removes most of the waiting code other tools need. Work that runs elsewhere — a custom thread pool, a network library Espresso does not know — must be registered as an idling resource, or the test will race it. Screens built with Jetpack Compose are tested with Compose’s own testing APIs, which follow the same principle.

@RunWith(AndroidJUnit4::class)
class LoginTest {
    @get:Rule
    val activityRule = ActivityScenarioRule(LoginActivity::class.java)

    @Test
    fun wrongPasswordShowsError() {
        onView(withId(R.id.email)).perform(typeText("layla.h@example.com"), closeSoftKeyboard())
        onView(withId(R.id.password)).perform(typeText("wrong-pass-1"), closeSoftKeyboard())
        onView(withId(R.id.sign_in)).perform(click())
        onView(withText(R.string.error_invalid_credentials)).check(matches(isDisplayed()))
    }
}

XCUITest: black-box through accessibility

An XCUITest test launches the app as a separate process and queries its element tree by type and identifier: buttons, text fields, static texts. Developers set an accessibilityIdentifier on each element the tests need; unlike the accessibility label, the identifier is not spoken by VoiceOver, so it can be a stable technical name. Launch arguments and environment variables put the app into a test state — a mock server URL, a reset data store, animations disabled — without special builds. Use waitForExistence(timeout:) for elements that appear after asynchronous work instead of fixed sleeps.

final class LoginUITests: XCTestCase {
    func testWrongPasswordShowsError() {
        let app = XCUIApplication()
        app.launchArguments += ["-uiTesting", "-resetState"]
        app.launch()

        let email = app.textFields["login.email"]
        email.tap()
        email.typeText("layla.h@example.com")

        let password = app.secureTextFields["login.password"]
        password.tap()
        password.typeText("wrong-pass-1")

        app.buttons["login.submit"].tap()

        XCTAssertTrue(app.staticTexts["login.error"].waitForExistence(timeout: 5))
    }
}

Appium: WebDriver for mobile

Appium is a server. Your test, written with a WebDriver client library, opens a session by sending capabilities — which platform, which automation driver, which app, which device — and then sends commands such as find element, click and send keys. Since Appium 2 the drivers are installed separately from the server, so a project states exactly which driver versions it uses. Capabilities that are not part of the W3C standard carry the appium: prefix. Because Appium drives the app from outside, the same test can run against a debug build on an emulator and a release candidate on a device-farm phone.

from appium import webdriver
from appium.options.android import UiAutomator2Options
from appium.webdriver.common.appiumby import AppiumBy
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

caps = {
    "platformName": "Android",
    "appium:automationName": "UiAutomator2",
    "appium:deviceName": "Pixel_Test",
    "appium:app": "/builds/shop-4.0.0-debug.apk",
    "appium:autoGrantPermissions": True,
}
driver = webdriver.Remote("http://127.0.0.1:4723", options=UiAutomator2Options().load_capabilities(caps))
try:
    driver.find_element(AppiumBy.ACCESSIBILITY_ID, "login.email").send_keys("layla.h@example.com")
    driver.find_element(AppiumBy.ACCESSIBILITY_ID, "login.password").send_keys("wrong-pass-1")
    driver.find_element(AppiumBy.ACCESSIBILITY_ID, "login.submit").click()
    WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((AppiumBy.ACCESSIBILITY_ID, "login.error")))
finally:
    driver.quit()

Locators on mobile

A locator is how a test finds an element, and on mobile the choice decides both stability and speed. Agree an identifier convention with developers (for example screen.element: login.submit), set identifiers on every element tests touch, and make "has test identifiers" part of the definition of done. The general principles are in the guide on locator strategies; the mobile specifics follow.

• Accessibility id — Appium’s preferred strategy. On iOS it matches the accessibilityIdentifier, which VoiceOver does not read, so it can be a technical name. On Android it matches the content description, which TalkBack does read aloud — so on Android it must be a real human label, or you should use the resource id instead.
• Resource id (Android) — for example com.example.shop:id/sign_in; stable as long as developers keep ids unchanged.
• Platform-native queries — UiSelector for UiAutomator2, predicate strings and class chains on iOS — when the above do not fit; faster than XPath.
• Visible text — breaks when the app is localized (Arabic and English builds show different text); use only with per-locale resources.
• XPath — last resort: slow on mobile because the whole view hierarchy has to be serialised, and fragile when layouts change.
• Cross-platform frameworks (React Native, Flutter) expose their own test-id property; check what it maps to on each platform before building locators on it.

Why mobile UI tests are flaky — and what to do

A flaky test passes and fails on the same code. Mobile adds its own causes to the usual ones (asynchrony, shared state, remote services, time), and a suite that fails randomly is soon ignored. The flaky tests guide covers detection and quarantine; these are the mobile-specific sources.

• Animations and transitions — disable them on test devices and emulators (the three animation scale settings below), or through launch arguments on iOS.
• Asynchrony the framework cannot see — register idling resources in Espresso, wait for conditions (element exists, is enabled) everywhere else; never use fixed sleeps.
• System dialogs — permission prompts, "app isn’t responding", keyboard tips, OS update prompts. Pre-grant permissions where the test is not about them, and handle expected system alerts explicitly.
• State leaking between tests — a signed-in session, local data, feature flags. Reset per test: clear app data (Android Test Orchestrator can clear package data between tests) or launch with a reset argument on iOS.
• Real network and a shared back end — use a mock server or a seeded test environment, and generate unique data per test run.
• The soft keyboard covering the next element, and elements that need scrolling into view.
• Device and emulator health — slow boots, full storage, low battery or overheating farm devices. Add health checks; retry at infrastructure level only, never to hide a real failure.
• Locale, timezone and clock differences between devices in the matrix.

# Android — turn off animations on a test device or emulator
adb shell settings put global window_animation_scale 0
adb shell settings put global transition_animation_scale 0
adb shell settings put global animator_duration_scale 0

Try it: Mock API Response Generator Try it: API Status Code Simulator Try it: Mock Error Response Generator Try it: Fake User Generator

Device farms and CI pipelines

Android emulators run on Linux, Windows or macOS CI machines and need hardware acceleration to be fast (KVM on Linux); iOS simulators require macOS. Split long suites into shards that run in parallel on several emulators or devices. A cloud device farm takes your app and test package, runs them on the real devices you select, and returns results with videos, screenshots and logs; the same devices are usually available for manual sessions. Choose farm devices from your device matrix, and use pairwise selection for the secondary parameters (language, orientation, font size) rather than multiplying runs. The tiers in the illustrative pipeline below are those of the example device matrix in the mobile testing pillar: Tiers 1 and 2 on every release, Tier 3 before major releases.

Publish results in a format the CI server understands (JUnit XML is the common one) and keep the artefacts of every failed test. A red build without a video and a device log is a build someone will re-run instead of fixing.

Example mobile pipeline

on every pull request
  1. unit tests (JVM / XCTest unit)
  2. build the app and the test package (test APK / UI test bundle)
  3. UI smoke: 10–20 critical tests on one Android emulator and one iOS simulator
  4. API contract tests against the staging back end
nightly
  5. full UI suite, sharded across emulators and simulators
  6. Tier 1 and Tier 2 real devices in a device farm
before release
  7. release candidate on the Tier 1 and Tier 2 devices (Tier 3 too before a major release)
     + exploratory session on real devices
  8. oldest supported app version against the new back end (API smoke)
for every failed test: video, screenshots, device log, JUnit XML

Try it: Pairwise Testing Generator Try it: API Schema Validator

What stays manual

Automation checks what you already know to ask. On mobile, several important questions are still answered best by a person with a real device: how gestures feel, whether a layout looks right on a device nobody scripted, what happens during a real phone call on a real network, and whether the app is usable with a screen reader. Automated accessibility checks exist on both platforms — Espresso can run accessibility checks during tests, and XCUITest can run an accessibility audit — and they catch missing labels, small touch targets and low contrast, but they do not replace a TalkBack or VoiceOver session.

• Exploratory sessions on real devices for each release candidate.
• Real interruptions: calls, Bluetooth, notifications from other apps.
• Visual review across devices, unless you maintain screenshot tests.
• Screen reader and large-text sessions.
• Battery, heat and performance on the lowest-end device.
• Store builds: installing from the store or the beta channel exactly as users do.

Try it: Test Data Factory

Frequently asked questions

References

• Appium DocumentationAppium (OpenJS Foundation)Official documentation for automating native, hybrid and mobile-web apps on iOS and Android.
• EspressoAndroid Developers (Google)Android's UI testing framework, which synchronises test actions with the app's UI thread automatically.
• XCUIAutomationAppleApple's API for UI tests that launch an app and interact with its elements the way a user does (the basis of XCUITest).
• XCTestAppleApple's framework for unit, performance and UI tests of Xcode projects on iOS and other Apple platforms.
• Test apps on AndroidAndroid Developers (Google)Android's official testing guidance: test types and scopes, local vs instrumented tests, and testing best practices.
• Eradicating Non-Determinism in Tests (Martin Fowler)martinfowler.com · 2011Why flaky tests destroy a suite's value, their common causes (isolation, asynchrony, remote services, time, resource leaks) and quarantine as a response.
• Certified Tester Advanced Level Test Automation Engineering (CTAL-TAE) v2.0ISTQB · 2024ISTQB's advanced syllabus on designing, implementing and maintaining test automation solutions and their architecture.
• Certified Tester Mobile Application Testing (CT-MAT)ISTQB · 2019ISTQB's specialist syllabus on mobile testing: app types, device and platform challenges, test types and tools.