An Android Device Farm with ADB + FFmpeg: Automated UI Testing Across 10–50 Devices

When your app has to work on a zoo of dozens of real phones, different vendors, Android versions, screen resolutions, manual testing quickly turns into a nightmare. Below is a Python pipeline that discovers every connected device on its own, installs the APK on all of them in parallel, runs instrumented tests, records a video of each run, and stitches those recordings into a single video report with FFmpeg.

The whole stack > Python + ADB + FFmpeg < is standard QA tooling. No magic, just automating the grind.

Pipeline architecture

1adb devices  ──►  list of serials
2     │
3     ▼
4APK install   (in parallel on all devices, ThreadPoolExecutor)
5     │
6     ▼
7for each device:
8     screenrecord (in background)  →  am instrument (run tests)  →  stop + pull video
9     │
10     ▼
11FFmpeg: overlay serial on each clip + concat  ──►  test_report.mp4

What you'll need

• ADB (Android Debug Bridge) from Android Platform Tools -> device control.
• Python 3.10+ -> orchestration (I use list[str], tuple[...] without from __future__).
• FFmpeg -> video processing and assembly.
• Devices with USB debugging enabled, connected over USB (or over Wi-Fi via adb tcpip).

One principle that runs through all the code: I pass arguments to subprocess as a list and without shell=True. It's safer (no injection through file names) and it doesn't break on spaces or special characters in paths.

1. Device discovery

adb devices also lists devices in the unauthorized / offline state. We keep only the ones actually in the device state.

1import subprocess
2
3def get_devices() -> list[str]:
4    """Return the serials of all devices in the 'device' state."""
5    out = subprocess.run(
6        ["adb", "devices"],
7        capture_output=True, text=True, check=True,
8    ).stdout
9
10    serials: list[str] = []
11    for line in out.splitlines()[1:]:      # first line is the "List of devices" header
12        line = line.strip()
13        if line.endswith("\tdevice"):      # drop unauthorized / offline
14            serials.append(line.split("\t")[0])
15    return serials

2. Parallel APK install

Installing on 50 devices one by one is slow. We spread the work across a thread pool: each adb install is a separate process, so threads work great here (we're waiting on I/O, not burning CPU).

1from concurrent.futures import ThreadPoolExecutor, as_completed
2
3def install_apk(serial: str, apk_path: str) -> tuple[str, bool, str]:
4    r = subprocess.run(
5        ["adb", "-s", serial, "install", "-r", "-g", apk_path],
6        capture_output=True, text=True,
7    )
8    ok = r.returncode == 0 and "Success" in r.stdout
9    return serial, ok, (r.stdout + r.stderr).strip()
10
11def install_on_all(apk_path: str, serials: list[str]) -> None:
12    with ThreadPoolExecutor(max_workers=len(serials) or 1) as pool:
13        futures = [pool.submit(install_apk, s, apk_path) for s in serials]
14        for f in as_completed(futures):
15            serial, ok, log = f.result()
16            print(f"[{'OK' if ok else 'FAIL'}] {serial}")
17            if not ok:
18                print(f"    {log}")

Flags: -r -> reinstall while keeping data, -g -> grant all runtime permissions immediately (handy so tests don't trip over permission dialogs).

3. Running instrumented tests

am instrument runs Espresso/JUnit tests on the device. It prints OK on success and FAILURES!!! on failure to stdout -> that's how we determine the result.

1def run_instrumented_tests(
2    serial: str,
3    package: str,
4    runner: str = "androidx.test.runner.AndroidJUnitRunner",
5) -> tuple[str, bool]:
6    r = subprocess.run(
7        ["adb", "-s", serial, "shell", "am", "instrument", "-w",
8         f"{package}/{runner}"],
9        capture_output=True, text=True,
10    )
11    ok = "FAILURES!!!" not in r.stdout and r.returncode == 0
12    return serial, ok

package is the test package ID, usually com.example.app.test.

4. Recording the screen during a test

screenrecord records video right on the device. Limitations to keep in mind: a ~3-minute cap per file and no audio. We start recording in the background, run the test, then stop it cleanly and pull the file to the host.

The most reliable way to stop the recording is not by signaling the local adb, but with pkill on the device itself -> that way screenrecord finalizes the MP4 container correctly.

1import time
2
3def start_recording(serial: str, remote: str = "/sdcard/run.mp4") -> subprocess.Popen:
4    return subprocess.Popen(
5        ["adb", "-s", serial, "shell", "screenrecord", remote]
6    )
7
8def stop_recording(
9    serial: str,
10    proc: subprocess.Popen,
11    remote: str = "/sdcard/run.mp4",
12    local: str = "run.mp4",
13) -> None:
14    # SIGINT on the device makes screenrecord close the file properly
15    subprocess.run(["adb", "-s", serial, "shell", "pkill", "-SIGINT", "screenrecord"])
16    proc.wait(timeout=10)
17    time.sleep(1)  # give the device a moment to finalize the container
18    subprocess.run(["adb", "-s", serial, "pull", remote, local], check=True)

5. Assembling the video report with FFmpeg

Different devices have different screen resolutions, so you can't just concatenate them with -c copy. We normalize each clip to a common format (1080×1920) and overlay the serial via drawtext along the way. After that all clips are identical and the final assembly is a fast concat with no re-encoding.

1def label_clip(src: str, dst: str, label: str) -> None:
2    """Scale to 1080x1920 and overlay a label (the device serial)."""
3    vf = (
4        "scale=1080:1920:force_original_aspect_ratio=decrease,"
5        "pad=1080:1920:(ow-iw)/2:(oh-ih)/2,"
6        f"drawtext=text='{label}':x=20:y=20:fontsize=42:"
7        "fontcolor=white:box=1:[email protected]"
8    )
9    subprocess.run(
10        ["ffmpeg", "-y", "-i", src, "-vf", vf,
11         "-an", "-c:v", "libx264", "-preset", "veryfast", "-crf", "23", dst],
12        check=True,
13    )
14
15def concat_report(clips: list[str], out: str = "test_report.mp4") -> None:
16    with open("concat_list.txt", "w") as f:
17        for c in clips:
18            f.write(f"file '{c}'\n")
19    subprocess.run(
20        ["ffmpeg", "-y", "-f", "concat", "-safe", "0",
21         "-i", "concat_list.txt", "-c", "copy", out],
22        check=True,
23    )

6. Putting it all together

1def main() -> None:
2    apk = "app-debug.apk"
3    package = "com.example.app.test"
4    runner = "androidx.test.runner.AndroidJUnitRunner"
5
6    serials = get_devices()
7    if not serials:
8        print("No devices found. Check USB and the output of 'adb devices'.")
9        return
10
11    print(f"Devices found: {len(serials)}")
12    install_on_all(apk, serials)
13
14    labeled: list[str] = []
15    for serial in serials:
16        proc = start_recording(serial)
17        _, ok = run_instrumented_tests(serial, package, runner)
18        stop_recording(serial, proc, local=f"{serial}.mp4")
19        print(f"[{'PASS' if ok else 'FAIL'}] tests on {serial}")
20
21        out = f"{serial}_labeled.mp4"
22        label_clip(f"{serial}.mp4", out, serial)
23        labeled.append(out)
24
25    concat_report(labeled, "test_report.mp4")
26    print("Done: test_report.mp4")
27
28if __name__ == "__main__":
29    main()

Here the "record + test" loop runs sequentially across devices -> it's more readable that way. For a real farm you'd want to wrap this block in a ThreadPoolExecutor too, so all devices are tested at once; the logic is the same as the install in section 2.

Don't reinvent the wheel: ready-made tools

• scrcpy -> real-time mirroring and control of a device from your PC. Indispensable when debugging failing tests.
• Appium / Espresso / UI Automator -> full-fledged UI testing frameworks; the am instrument above is their engine.
• Gradle Managed Devices -> run tests on emulators straight from the build, no manual ADB wrangling.
• Firebase Test Lab / AWS Device Farm -> a cloud fleet of real devices if you'd rather not keep your own hardware.
• GNU parallel -> if you'd rather orchestrate from bash than Python.

The economics: what it costs and what it saves

Test automation isn't about "making money out of thin air" -> it's about cutting the two most expensive line items: person-hours and cloud minutes. Below is an estimate across three typical scales. The numbers are illustrative and depend on region, device vendor, and provider pricing, check current rates before buying.

Your own farm -> one-time investment

Item

For 10 devices

For 30 devices

For 50 devices

Used Android phones (~$60 each)

~$600

~$1,800

~$3,000

Powered USB hubs

~$100

~$250

~$400

Mini-PC / host

~$400

~$400

~$500

Cables, rack, odds and ends

~$80

~$150

~$250

One-time total

~$1,200

~$2,600

~$4,150

Electricity per month

pennies

~$10–20

~$20–40

This is capital expenditure: pay once, and the farm then runs for years at almost no cost.

Cloud -> you pay per minute

Firebase Test Lab, AWS Device Farm, BrowserStack, and the like charge per device-minute -> roughly $0.05–0.20 per device-minute. A single regression run on 30 devices at 5 minutes each is 150 device-minutes, i.e. ~$7.5–30 per run.

Now multiply by CI intensity:

Run frequency

Runs per month

Cost (at ~$15/run)

2× per day

~44

~$660/mo

10× per day

~220

~$3,300/mo

On every push (active team)

500+

$7,500+/mo

Break-even: a 30-device farm (~$2,600) pays for itself against the cloud in roughly 4 months at a modest 2 runs per day; with active CI, in under a month. After that the cloud bill keeps dripping every month, and the farm doesn't.

Manual labor -> what gets freed up

A manual run of a single regression scenario on 30 devices is on the order of a QA engineer's workday. Running regression twice a week adds up to ~8 person-days a month. At a QA cost of somewhere around $1,600–2,700/mo, that's a meaningful chunk of a salary that the pipeline frees up for meaningful work instead of the "connect–install–poke–record" grind ×30.

How this converts into money

There's no direct income "from the script" here -> there are three indirect but very real mechanisms:

• Faster releases. Regression in minutes instead of a day → you ship features more often → you respond to the market faster. For a subscription product, that ties directly into retention and revenue.
• Fewer bugs in production. Catching a crash on a specific Samsung before release costs pennies; the same crash reaching users means dropped store ratings, churn, and refunds. Every bug caught early is a handful of negative reviews that never get written.
• It sells as a service. Setting up a device farm and CI testing is an in-demand role in freelancing and outsourcing. The pipeline above is a ready-made core for such an offering.

The key difference from "engagement-farming schemes": there, the money comes from deceiving algorithms and ends in a ban. Here, it comes from saved hours and prevented losses. The first falls apart; the second is a sustainable business case you're not ashamed to show a client.

Wrap-up

The result is a reproducible pipeline: one command, and the app is exercised across the entire device fleet, with a video report at the end that shows the behavior on each specific device.

Bookmark and subscribe so that you don't miss useful articles that will save money 📝