View item

DARPA – Systematizing Confidence in Open Research and Evidence (SCORE) - HR001118S0047
Sponsor: US Defense Advanced Research Projects Agency (DARPA) - Defense Sciences Office (DSO) 		Closing Date: 12-Mar-2019
DARPA is soliciting innovative research proposals for the development and deployment of automated tools to assign Confidence Scores (CSs) to different kinds of Social and Behavioral Science (SBS) research results and claims. CSs are quantitative measures that should enable someone to understand the degree to which a particular claim or result is likely to be reproducible and/or replicable. These tools will assign explainable CSs with a reliability that is equal to, or better than, the best current human expert methods and will enable a consumer of SBS research to quickly calibrate the level of confidence in the Reproducibility and Replicability (R&R) of a given SBS result or claim. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.

The vision of the SCORE program is to test, validate, and demonstrate the feasibility and utility of one or more automated tools for assigning CSs to a wide range of SBS claims. SCORE seeks to realize this vision through a two-phase program. The first Phase will focus on developing the initial Common Task Framework (CTF) for SCORE, with a curated dataset of SBS research claims using methods for rapidly but accurately labeling those data with human expert CSs. Early algorithm development will also occur in Phase 1 as proof of principle. In Phase 2, performers will use those labeled data to train and test algorithms that will assign quantitative CSs. These algorithm-based CSs will be compared to CSs assigned by the best performing human expert methods to see how they overlap. If successful, this program will enable SBS consumers within the DoD and the U.S. Government to use SCORE algorithms to quickly, accurately, and iteratively calibrate the confidence they should have in a particular SBS claim’s R&R. SCORE deliverables should have significant positive impact on DoD and USG’s abilities to leverage SBS for modeling, planning for, and operating in, the Human Domain.

The SCORE program will fund research in three Technical Areas (TAs), with an independent Test and Evaluation (T&E) team providing oversight. The three TAs are: TA1: Data; TA2: Experts; and TA3: Algorithms. Proposals for TA3 only are currently open.

Successful TA3 teams will develop SCORE algorithms using training data provided by TA1 (as well as other training data they may propose) to automatically assign CSs to held-out TA1 test sets. TA3 team(s) will be responsible for all aspects of algorithm development. This responsibility extends to all essential feature identification, extraction, and engineering steps, whether carried out over TA1-curated data or over TA3-provided data. TA3 algorithms will be evaluated in terms of their CSs overlap with the best performing TA2 CSs, ultimately seeking to achieve 95% confidence in an algorithm’s overlap with those TA2 CSs. Towards the end of Phase 2, successful TA3 teams will also demonstrate the utility of their algorithms/systems for DoD users and applications.

Key Dates
TA3 Abstracts close with DARPA: 31 January 2019
FAQ Submission Deadline for TA3: 18 February 2019
TA3 Full Proposals close with DARPA: 12 March 2019

Overview

• Eligibility
Non-U.S. organizations and/or individuals may participate to the extent that such participants comply with any necessary nondisclosure agreements, security regulations, export control laws, and other governing statutes applicable under the circumstances.

• Funding
The level of funding for individual awards will depend on the quality of the proposals received and the availability of funds.

How to Apply

Step 1. Abstract
Prior to submitting a full proposal, proposers are strongly encouraged to submit an abstract. DARPA will respond to abstracts with a statement as to whether DARPA is interested in the idea. If DARPA does not recommend the proposer submit a full proposal, DARPA will provide feedback to the proposer regarding the rationale for this decision. Regardless of DARPA’s response to an abstract, proposers may submit a full proposal.

Step 2. Full Proposal
Full proposals are submitted via Grants.gov. Interested applicants are strongly encouraged to contact the UQR&I International Team at internationalgrants@research.uq.edu.au well in advance of the abstract closing date for information on the application process. The Broad Agency Announcement (BAA) is available from Grants.gov.
Website: https://www.grants.gov/web/grants/view-opportunity.html?oppId=306186

Return