NIST RFQ: Genomics Privacy-Preserving Federated Learning Platform Project Support Services

Notice ID: 1333ND24QNB770554

The Contractor shall complete the following tasks:

Task 1: Create a suite of empirical privacy evaluation metrics.

1. Identify an extensive list of candidate empirical privacy metrics from academic literature, existing software repositories, and other relevant sources.
2. Recruit a set of at least five subject matter experts (SMEs) to advise the selection, evaluation, and implementation of privacy metrics.
3. Work with NIST staff to select a set of metrics to implement as original software. Develop a prototype of the metrics as a Python software library with the capability of examining the outputs of the NIST Genomics PPFL Platform.
4. Implement the prototype software from Specification 3 into the SDNist software library and configure the metrics to evaluate existing CRC data.
5. Create a presentation to communicate the project goals and recruit participants, partners, and other stakeholders at workshops, meetings, conferences, etc.

Task 2: Conduct a community red-teaming exercise to simulate privacy attacks on the Genomics PPFL Platform and the CRC deidentified datasets.

1. Work with NIST staff to develop a project plan for a community red-teaming exercise against the Genomics PPFL Platform and the CRC data. The project plan must have clear objectives and milestones that align to the due dates specified in this statement of work. Create a presentation to communicate the project goals and recruit participants, partners, and other stakeholders at workshops, meetings, conferences, etc.
2. Create a Python library to evaluate the fidelity and utility of the Genomics PPFL Platform output models. NIST and its delegates will collaborate to select a set of data fidelity and utility metrics. The test software shall output a standardized report in machine- and human-readable formats suitable for downstream analysis and research.
3. Develop a Python library to serve as a test harness to conduct the red-teaming exercise on the Genomics PPFL Platform. The test harness must be capable of accepting and validating community-created simulated privacy attacks in the form of Python scripts. The test harness must have an exemplar privacy attack script. The test harness must have clear documentation to facilitate community participation. The test harness will use the privacy metrics developed in Task 1 and output a standardized report in machine- and human-readable formats …

Task 3: Conduct red-teaming exercises on the Genomics PPFL Platform and the CRC data archive.

Coordinate a red-teaming exercise for the Genomics PPFL Platform models. The Contractor will develop a method to receive and validate privacy attack Python scripts from the privacy and data science community. The Contractor will run the attack scripts against the Genomics PPFL Platform models and collect the resulting reports. The Contractor shall create a report detailing participation and outcomes.

Task 4: Complete a final report that summarizes the major activities of this project.

The report shall include a directory with links to all the major resources developed during the project. The report shall correlate the results of the empirical privacy evaluation and the red-teaming process with the goal of guiding which empirical evaluations best represent privacy attacks. The report shall include detailed summary statistics on the privacy, fidelity, and utility evaluations made during the project.

Task 5: Maintain the content of the Genomics PPFL Platform evaluation library, the CRC website, and the CRC Github repositories …

The period of performance shall be 12 months, inclusive of Optional Tasks.

Read more here.

🔊 Listen to Story