Why mobile test automation estimates are usually wrong

Most mobile test automation estimates are wrong for the same reason: they are made before anyone has opened the accessibility tree.

On a mobile app for an international sports federation, we planned the automation the sensible way. Pick the flows, pick the stack, size the work. Then we started, and the estimate met the platform.

Where the estimate met the platform

Three things cost more than the plan allowed for. None of them were visible until we tried to automate a real screen on a real device.

A control with no identity

A dropdown built in React Native rendered fine and read fine to a human, but on iOS it exposed almost nothing to the accessibility tree. No stable identity to hold on to. The test could see the screen and could not name the thing on it.

Mobile automation tools find elements through what the platform exposes: identifiers, labels, roles. When a component does not expose them, the alternatives are all bad. Screen coordinates break on the next device size. Long structural paths break on the next layout change. Image matching breaks on the next theme.

System UI on its own schedule

Biometric prompts were worse. They are system UI, not part of the app, and they arrive on their own schedule. A test that assumes a fixed moment passes on your laptop and fails on the device farm. The same is true of permission dialogs and notifications: anything the operating system owns has to be handled as an event, not a step.

Two platforms, two behaviours

And the same flow behaved differently enough on iOS and Android that write once was never really on the table. Not different screens, just different enough timing, focus and element structure that a shared test needed a branch at almost every step.

None of this is exotic

This is the ordinary state of mobile automation, and it is completely invisible from a planning document. A list of user flows tells you what to test. It tells you nothing about whether each control on those flows can be reliably found by a test.

That is the real unknown in the estimate. An automation estimate is not a guess about effort. It is a guess about what the platform will let you address.

The fix is often an application change

The usual answer to a control with no identity is to give it one, by adding a stable identifier or an accessibility label in the application code. That is a small change, but it is a change to the product, made by the product team, on their schedule.

This is the dependency most estimates miss. The test team cannot fix it alone, so the estimate has to include the conversation, the change and the release that carries it. A useful side effect: much of that work also improves what assistive technology sees, so it is rarely a hard sell once it is visible.

The cheapest day in the project

So before you commit to a number, spend a day on the screens the app is actually about.

1. 1.Pick the two or three screens that carry the product: the ones that make money, or the ones users open every day.
2. 2.Automate one of them end to end, on both platforms, on real devices or the device farm you will actually use.
3. 3.Inspect the accessibility tree for every control on those screens, and list the ones with no stable identity.
4. 4.Note every interaction with system UI: biometrics, permissions, notifications, deep links.
5. 5.Re-estimate with that list in front of you, and raise the application changes with the product team now.

You will either confirm the estimate or find out which control has no identity. Either result is worth more than the day it costs.

Agents hit the same wall

This matters more, not less, as more test code is written by coding agents. An agent can produce a mobile test quickly, but it can only address what the platform exposes. Faced with a control that has no identity, it will do what a hurried engineer does: fall back to coordinates or a brittle path, and produce a test that passes once and fails on the next device.

So the accessibility layer of a mobile app is part of its readiness for agentic delivery. A team that has given its controls stable identities can let agents write and maintain tests with some confidence. A team that has not will get fast tests that do not last.

An automation estimate is not a guess about effort. It is a guess about what the platform will let you address.