![]() Each of these models is tuned for high precision so that the union of their outputs achieves (high) recall objectives. ![]() Second, rather than develop one monolithic prediction model, EMBERS takes the approach of developing multiple machine learning models. 2 In particular, EMBERS is built on a simple message-passing design that requires few shared dependencies between components and no shared infrastructure. To address these needs, the system is composed of many simple independent components strung together in a pipes-and-filters architecture. Additionally, the vast majority of the processing is performed on continuous streams of data that need to be processed in near-real-time. This diverse team required a highly distributed and loosely coupled functional architecture, allowing team members to develop components for the system without worrying about dependencies among them. The team composition involves eight research universities and two industry partners contributing diverse expertise in computer science, machine learning, disease modeling, social science, linguistic processing, and systems integration. First, the EMBERS system architecture was designed to support collaboration from the outset. Three key considerations motivated the design of EMBERS. “TO ADDRESS THESE NEEDS, THE SYSTEM IS COMPOSED OF MANY SIMPLE INDEPENDENT COMPONENTS STRUNG TOGETHER IN A PIPES-AND-FILTERS ARCHITECTURE.” Much of the system is designed to look for precursor signals in social media streams and use these indicators to drive statistical and machine learning algorithms that generate the predictions. The system processes a range of data, from high-volume, high-velocity, noisy open-source media such as Twitter to lower-volume, higher-quality sources, such as economic indicators. It has been operational and delivering predictions (and continues to) since November 2012. For civil unrest, EMBERS produces detailed forecasts about future events, including the date, location (to within a city resolution), type of event (e.g., whether it is a protest for wages or a protest for safety), and protesting population (e.g., educators, factory workers, doctors), along with uncertainties involved in the forecasts. The classes of events EMBERS is designed to forecast include influenza-like illness case counts, rare disease outbreaks, elections, domestic political crises, and civil unrest (we focus in this article primarily on civil unrest). EMBERS is supported by the Intelligence Advanced Research Project Activity (IARPA) Open Source Indicators (OSI) program. Early Model Based Event Recognition using Surrogates (EMBERS) 1 is an anticipatory intelligence system for forecasting socially significant population-level events, such as civil unrest incidents, disease outbreaks, and election outcomes, on the basis of publicly available data. One of the promising themes in this space is the idea of harnessing open-source datasets to identify threats and support decision making for national security, law enforcement, and intelligence missions. We also present a detailed prospective evaluation of EMBERS in forecasting significant societal events in the past 2 years.Ī nticipatory intelligence is considered to be one of the next frontiers of “big data” research, wherein myriad data streams are fused together to generate predictions of critical societal events. We describe the architecture of the system, some of the design tradeoffs encountered during development, and specifics of the machine learning models underlying EMBERS. It is deployed on Amazon Web Services using an entirely automated deployment process. ![]() EMBERS is built on a streaming, scalable, loosely coupled, shared-nothing architecture using ZeroMQ as its messaging backbone and JSON as its wire data format. It has been operational since November 2012 and delivers approximately 50 predictions each day for countries of Latin America. Developed under the Intelligence Advanced Research Project Activity Open Source Indicators program, Early Model Based Event Recognition using Surrogates (EMBERS) is a large-scale big data analytics system for forecasting significant societal events, such as civil unrest events on the basis of continuous, automated analysis of large volumes of publicly available data. ![]()
0 Comments
Leave a Reply. |