What is it about?
Partial observations are common in datasets used for machine learning, due to reasons ranging from human error to data corruption due to time delays and sensor failure. In modern data-driven applications, practitioners are often divorced from the data generation process, so do not have accurate models of the causes underlying the missingness. In turn, practitioners are often tempted to assume that the missingness is benign, in exchange for simplified imputation and analysis procedures. This work introduces Adversarial Missingness (AM) as a new threat model that exploits the inherent difficulty of accurately modeling unknown missingness mechanisms to attack machine learning models. In AM, an attacker is limited to omitting entries from a dataset, as opposed to adding or changing samples. By crafting malicious missingness mechanisms, the adversary can decrease the trustworthiness of learning algorithms by making them less accurate, biased, and unfair. To demonstrate the feasibility of AM attacks, this paper focused on Causal Machine Learning (ML), which provides advances over standard ML in fairness, generalization, and adversarial robustness. We demonstrate that it is possible to design adversarial missingness mechanisms by omitting certain entries of fully observed datasets in order to hide causal relationships and manipulate learning outcomes.
Featured Image
Photo by Claudio Schwarz on Unsplash
Why is it important?
First, the success of AM points to an important and exploitable distinction between trust in the correctness of the observed data and trust in the assumptions used in learning from that data. Standard data poisoning defenses, including cryptographically signed input validation, are effective against attacks on the trustworthiness of the observed data (i.e., they defend against arbitrary modification or addition of inputs). But there are currently no effective defense mechanisms against AM attacks, which violate the assumptions implicitly made by methods used to learn from partially observed data. Second, in causal modeling, a single change like adding or removing an edge can have profound effects: imagine, for instance, the impact on public policy of removing the causal link between exposure to PCBs and cancer. AM attacks can lead to targeted alterations in the inferred causal structures that create bias and fairness issues in downstream applications.
Perspectives
We focused on casual machine learning since it is arguably more suitable for trusted AI/ML systems; however, AM attacks are applicable to correlation-based ML models as well, with similar adversarial effects. AM cannot be prevented by general stateless cryptographic signatures on input items, and standard data poisoning defenses are inapplicable to many techniques for learning from partially observed data. Defending against AM attacks seems to require that we significantly modify current techniques used for learning from partially observed data to exhibit robustness to the observed missingness pattern, and move away from learning algorithms that completely trust the observed missingness pattern (even when the observed data can be trusted, e.g. because it is cryptographically authenticated).
Deniz Koyuncu
Rensselaer Polytechnic Institute
Read the Original
This page is a summary of: Adversarial Missingness Attacks on Causal Structure Learning, ACM Transactions on Intelligent Systems and Technology, August 2024, ACM (Association for Computing Machinery),
DOI: 10.1145/3682065.
You can read the full text:
Contributors
The following have contributed to this page
