Machine studying fashions have been educated to foretell semantic details about consumer interfaces (UIs) to make apps extra accessible, simpler to check, and to automate. Presently, most fashions depend on datasets which can be collected and labeled by human crowd-workers, a course of that’s expensive and surprisingly error-prone for sure duties. For instance, it’s potential to guess if a UI aspect is “tappable” from a screenshot (i.e., based mostly on visible signifiers) or from doubtlessly unreliable metadata (e.g., a view hierarchy), however one solution to know for sure is to programmatically faucet the UI aspect and observe the results. We constructed the Endless UI Learner, an app crawler that robotically installs actual apps from a cell app retailer and crawls them to find new and difficult coaching examples to study from. The Endless UI Learner has crawled for greater than 5,000 device-hours, performing over half one million actions on 6,000 apps to coach three pc imaginative and prescient fashions for i) tappability prediction, ii) draggability prediction, and iii) display similarity.