Contemporary deployment environments are volatile, with conditions that are often hard to predict in advance, demanding solutions that are able to learn how best to design a system at runtime from a set of available alternatives. While the self-adaptive systems community has devoted significant attention to online learning, there is less research specifically directed towards learning for open-ended architectural adaptation - where individual components represent alternatives that can be added and removed dynamically. In this paper we present the Emergent Web Server (EWS), an architecture-based adaptive web server with 42 unique compositions of alternative components that present different utility when subjected to different workload patterns. This artefact allows the exploration of online learning techniques that are specifically able to consider the composition of logic that comprises a given system, and how each piece of logic contributes to overall utility. It also allows the user to add new components at runtime (and so produce new composition options), and to remove existing components; both are likely to occur in systems where developers (or automated code generators) deploy new code on a continuous basis and identify code which has never performed well. Our exemplar bundles together a fully-functional web server, a number of pre-packaged online learning approaches, and utilities to integrate, evaluate, and compare new online learning approaches.