Harms in Machine Learning
A practical reflection on sources of harm across the machine learning lifecycle and how design choices affect people.
Understanding Potential Sources of Harm throughout the Machine Learning Life Cycle
The case study outlines sources of harm throughout the machine learning lifecycle, including data collection, modeling, evaluation, and deployment. I found its typology useful because it connects technical choices to specific forms of harm.
I have seen related issues in practice. During my nonprofit work, job seekers sometimes faced measurement bias when employability was inferred from U.S. education or formal experience. Deployment bias appeared when recruiters or systems gave model outputs more authority than they deserved. Aggregation bias appeared when equal rules ignored unequal needs, such as applying the same assistance limit to clients with very different transportation, family, or financial constraints.
Representation bias also appeared in survey work. Clients who visited frequently were easier to reach, while people with less time, inconsistent schedules, or limited technology access were less likely to respond. That can distort organizational data and influence future decisions.
Qualitative research can help address these gaps by revealing subgroups and lived experiences that aggregate metrics miss. Before measuring a concept such as employability, risk, or success, designers should ask whether the concept itself is valid and ethical.
A harm-prevention checklist should begin by defining the decision, the affected stakeholders, and the outcomes that matter. It should then ask whether labels are proxies, whether performance varies by subgroup, whether the development and deployment populations differ, and whether affected people have opportunities to give feedback or appeal.
The question this raises for me is: can transparency alone correct algorithmic harm?
Transparency is necessary but insufficient. Documentation can reveal bias, but it does not automatically change power relations or outcomes. Effective governance also needs appeal processes, participatory review, and the ability to challenge data-driven decisions.
This reading sharpened my vocabulary for describing bias. It also reframed my nonprofit experience: data work is not only about who receives resources, but also about whose experiences are visible in the first place.