Lecture
Это окончание невероятной информации про алгоритмическая предвзятость.
...
secrets protects companies such as search engines, where a transparent algorithm could reveal tactics for manipulating search rankings.: 366 This makes it difficult to conduct research or analysis to determine how algorithms work.: 20 Critics suggest that such secrecy may also conceal possibly unethical methods used in creating or processing algorithmic results.: 369 Other critics, such as lawyer and activist Katarzyna Szymielewicz, suggest that a lack of transparency is often masked by the complexity of algorithms, protecting companies from having to disclose or having their own algorithmic processes investigated.
A significant barrier to understanding how to address bias in practice is that categories such as the demographic data of persons protected by anti-discrimination legislation are often not explicitly accounted for during data collection and processing. In some cases, the ability to explicitly collect such data is limited, for example, in device identification, ubiquitous computing and the Internet of Things. In other cases, a data controller may be unwilling to collect such data for reputational reasons, or because it represents increased liability and security risk. It may also be the case that, at least with respect to the European Union's General Data Protection Regulation, such data falls under the “special category” provisions (Article 9) and therefore entails more restrictions on potential collection and processing.
Some specialists have attempted to estimate and fill in these missing sensitive categories in order to reduce bias, for example, by creating systems to determine ethnicity from names, although this can lead to other forms of bias if not approached with caution. Machine learning researchers have used privacy-enhancing cryptographic technologies, such as secure multi-party computation, to propose methods by which algorithmic bias can be assessed or reduced without disclosing this data to model developers in the clear.
Algorithmic bias involves not only protected categories but can also concern characteristics that are less easily observed or codified, such as political views. In these cases, an easily accessible or uncontroversial ground truth rarely exists, and removing bias from such a system becomes more difficult. In addition, false and spurious correlations can arise from an incomplete understanding of protected categories, for example, insurance rates based on historical car accident data that may, purely by coincidence, coincide with the residential clusters of ethnic minorities.
Among the main approaches to overcoming algorithmic bias are auditing algorithms and testing them on different data subgroups, collecting and cleaning training sets for balance, using transparent or open models with the ability to explain the decisions made, and developing ethical and legal standards for the application of AI.
A study of 84 guiding principles for ethical AI policy found that fairness and “mitigating unwanted bias” are a common point of concern, addressed through a combination of technical solutions, transparency and monitoring, the right to remedy and enhanced oversight, and efforts to ensure diversity and inclusion.
Several attempts have been made to create methods and tools capable of detecting and monitoring bias in an algorithm. These new fields focus on tools that are typically applied to the (training) data used by the program rather than to the algorithm's internal processes. These methods can also analyze the program's output and its usefulness, and may therefore include analysis of its error matrix (or confusion table). Explainable AI for detecting algorithmic bias is a proposed method for detecting the presence of bias in an algorithm or learning model. Using machine learning to detect bias is called “conducting an AI audit,” where the “auditor” is an algorithm that runs through the AI model and training data to identify biases. Ensuring that an AI tool, such as a classifier, is free of bias is more difficult than simply removing sensitive information from inputs, since it is usually implied by other signals. For example, a job candidate's hobbies, sports and the schools they attended can reveal their gender to the software even if that information is removed from the analysis. Solutions to this problem include ensuring that an intelligent agent possesses no information that could be used to reconstruct protected and sensitive information about the subject, as was first demonstrated in a study where a deep learning network was simultaneously trained to perform a task while remaining completely independent of the protected characteristic. A simpler method was proposed in the context of word embeddings and involves removing information that correlates with the protected characteristic.
A new IEEE standard is currently [ when? ] being developed, aimed at defining methodologies to help algorithm developers address bias issues and ensure transparency (i.e., for authorities or end users ) regarding the operation and possible consequences of their algorithms. The project was approved in February 2017 and is sponsored by the Software and Systems Engineering Standards Committee, a committee created by the IEEE Computer Society. The draft standard was expected to be put to a vote in June 2019. The standard was published in January 2025.
In 2022, IEEE released a standard aimed at defining methodologies to help algorithm creators address bias issues and promote transparency regarding the functioning and potential effects of their algorithms. The project, initially approved in February 2017, was sponsored by the Software and Systems Engineering Standards Committee, a committee of the IEEE Computer Society. The standard provides guidelines for ensuring transparency for authorities or end users and for mitigating algorithmic bias.
AI ethics guidelines point to the need for accountability, recommending that steps be taken to improve the interpretability of results. Such solutions include consideration of a “right to understanding” in machine learning algorithms and resistance to deploying machine learning in situations where decisions cannot be explained or reviewed. To this end, organizations such as DARPA are already pursuing a movement toward “explainable AI” for reasons extending beyond bias mitigation. For example, Price Waterhouse Coopers also suggests that monitoring outcomes means designing systems in such a way as to ensure that individual system components can be isolated and shut down if they are found to skew results.
An initial approach to transparency involved open-sourcing algorithms. Software code can be examined and improvements proposed using source code hosting tools. However, this approach does not necessarily produce the desired outcome. Companies and organizations can share all possible documentation and code, but this does not ensure transparency if the audience does not understand the information provided. The role of an interested, critical audience therefore deserves examination in the context of transparency. Algorithms cannot be held accountable without a critical audience.
Several approaches to documentation have been proposed to increase transparency and support bias assessment in algorithmic systems. One widely cited method is the use of model cards, which provide standardized summaries of an AI system's intended use, performance metrics, evaluation datasets, and known limitations. Related efforts include datasheets for datasets, which describe the provenance, composition, collection methods, and recommended uses of training data. These documentation frameworks are intended to clarify the assumptions and potential biases embedded in training data and machine learning systems, helping practitioners, auditors, and stakeholder groups better interpret system behavior.
In addition to documentation practices, researchers and policymakers encourage the use of structured governance mechanisms, such as algorithmic impact assessments, risk-based evaluation procedures, and post-deployment monitoring. These processes aim to identify potentially disparate impacts before deployment and to ensure that AI systems continue to be evaluated for fairness during real-world operation. Public-sector initiatives, such as Canada's Directive on Automated Decision-Making, require impact assessments, explainability measures, and regular audits for certain high-risk automated systems. Together, these governance approaches complement technical mitigation strategies, ensuring accountability and transparency throughout the AI development and deployment lifecycle.
From a regulatory standpoint, the Toronto Declaration calls for applying a human rights framework to harm caused by algorithmic bias. This includes legislating due-diligence expectations for the developers of these algorithms and establishing accountability in cases where private actors fail to protect the public interest, noting that such rights may be obscured by the difficulty of assigning responsibility within a network of complex, interconnected processes. Others suggest the need for clear liability insurance mechanisms.
Amid concerns that the development of AI systems is largely the preserve of white male engineers, a number of scholars have suggested that algorithmic bias can be minimized by broadening inclusivity among those who develop AI systems. For example, only 12% of machine learning engineers are women, and Black leaders in AI point to a “diversity crisis” in the field. Groups such as Black in AI and Queer in AI are attempting to create more inclusive spaces within the AI community and push back against the often harmful priorities of corporations that control the trajectory of AI research. Criticism of simplistic inclusivity efforts suggests that diversity programs cannot address intersecting forms of inequality, and calls for a more targeted approach that accounts for intersectionality in algorithm design.: 4 Researchers at the University of Cambridge argue that addressing racial diversity is complicated by the “whiteness” of AI culture.
Integrating interdisciplinarity and collaboration into the development of AI systems can play a decisive role in combating algorithmic bias. Integrating knowledge, expertise, and perspectives from disciplines outside computer science can foster a better understanding of how data-driven decisions affect society. An example of this in AI research is PACT, or Participatory Approach to enable Capabilities in Communities, a proposed framework for fostering collaboration in the development of AI-based solutions oriented toward social impact. This framework sets out guiding principles for stakeholder participation when working on AI for social good (AI4SG) projects. PACT attempts to emphasize the importance of decolonization efforts and the redistribution of power in the development of human-centered AI solutions. An academic initiative in this regard is Stanford University's Institute for Human-Centered Artificial Intelligence, which seeks to foster interdisciplinary collaboration. The institute's mission is to advance research, education, policy, and practice in artificial intelligence (AI) to improve the human condition.
Collaboration with outside experts and diverse stakeholders fosters the ethical, inclusive, and responsible development of intelligent systems. It incorporates ethical considerations, accounts for social and cultural context, promotes human-centered design, draws on technical expertise, and addresses policy and legal aspects. Collaboration across disciplines is essential for effectively reducing bias in AI systems and ensuring that AI technologies are fair, transparent, and accountable.
The General Data Protection Regulation (GDPR), the European Union's revised data protection regime that came into effect in 2018, addresses in Article 22 “automated individual decision-making, including profiling.” These rules prohibit “solely” automated decisions that have a “significant” or “legal” effect on a person, unless explicitly authorized by consent, contract, or member state law. Where they are permitted, safeguards must be in place, such as the right to obtain human intervention in the decision-making process and a non-binding right to an explanation of decisions made. Although these rules are often considered new, almost identical provisions have existed in Europe since 1995, in Article 15 of the Data Protection Directive. The original rules and safeguards for automated decision-making have existed in French law since the late 1970s.
The GDPR addresses algorithmic bias in profiling systems, as well as the statistical approaches that can be used to correct for it, directly in Recital 71, noting that
The controller should use appropriate mathematical or statistical procedures for the profiling, implement appropriate technical and organisational measures... that prevent, in particular, discriminatory effects on natural persons on the basis of racial or ethnic origin, political opinion, religion or beliefs, trade union membership, genetic or health status or sexual orientation, or that result in measures having such an effect.
As with the non-binding right to explanation in Recital 71, the problem lies in the non-binding nature of the recital. Although this was treated as a requirement by the Article 29 Working Party, which advised on the implementation of data protection legislation, its practical aspects remain unclear. It has been argued that data protection impact assessments for high-risk data profiling (along with other preventive measures within data protection) may be a better way of addressing algorithmic discrimination problems, since they constrain the actions of those who use algorithms rather than requiring consumers to file complaints or request changes.
The United States has no comprehensive legislation regulating algorithmic bias, and the issue is addressed through various state and federal laws that may vary by industry, sector, and the manner in which an algorithm is used. Many policies are self-implemented or overseen by the Federal Trade Commission. In 2016, the Obama administration released the National Artificial Intelligence Research and Development Strategic Plan, which was intended to help policymakers critically evaluate algorithms. It recommended that researchers “design these systems so that their actions and decision-making are transparent and easily interpretable by humans, and thus can be examined for any bias they may contain, rather than simply being absorbed and repeating these biases.” Intended only as guidance, the report did not create any legal precedent.: 26
In 2017, New York enacted the first algorithmic accountability law in the United States. The law, which took effect on January 1, 2018, required “the establishment of a task force that will provide recommendations on how information on agency automated decision systems may be shared with the public, and how agencies may address instances where people are harmed by agency automated decision systems.” In 2023, New York passed a law requiring employers using automated hiring tools to conduct independent “bias audits” and publish the results. This law became one of the first legislatively mandated measures to ensure transparency regarding AI systems used in hiring decisions in the United States. The task force was required to submit its findings and recommendations for further regulatory action in 2019. On February 11, 2019, under Executive Order 13859, the federal government introduced the “American AI Initiative,” a comprehensive strategy for maintaining U.S. leadership in artificial intelligence. This initiative emphasizes the importance of sustained research and development in AI, ethical standards, workforce training, and the protection of critical AI technologies. This aligns with broader efforts to ensure transparency, accountability, and innovation in AI systems across the public and private sectors. In addition, on October 30, 2023, the president signed Executive Order 14110, which emphasizes the safe, secure, and trustworthy development and use of artificial intelligence (AI). The order outlines a coordinated, whole-of-government approach to harnessing the potential of AI while reducing its risks, including fraud, discrimination, and threats to national security. An important focus of this commitment is fostering responsible innovation and cross-sector collaboration to ensure that AI benefits society as a whole. With this order, President Joe Biden directed the federal government to develop best practices for companies to optimize the benefits of AI while minimizing its harms.
On July 31, 2018, a draft Personal Data Protection Bill was introduced. The bill proposes standards for the storage, processing, and transfer of data. Although it does not use the term “algorithm,” it includes provisions on “harm arising from any processing or any kind of processing undertaken by a fiduciary.” It defines “any denial or withdrawal of a service, benefit, or good resulting from an evaluative decision about the data subject” or “any discriminatory treatment” as a source of harm that may arise from the improper use of data. It also contains special provisions for people with “intersex status.”
Часть 1 Algorithmic Bias
Часть 2 Mitigation approaches - Algorithmic Bias
Comments