Artificial intelligence systems used in recruitment may reproduce existing biases, analyse an excessively broad range of information about applicants and contribute to discriminatory decisions. Mirosław Wróblewski, President of Poland’s Personal Data Protection Office, known as UODO, has asked the Ministry of Family, Labour and Social Policy to begin work on legislation providing additional protection for job applicants and employees.
UODO is receiving a growing number of reports about the use of artificial intelligence tools in recruitment processes. Such systems may be used to analyse application documents, shortlist candidates, create applicant profiles and support decisions on whether someone should be invited to an interview or offered a job.
According to the President of UODO, the development of these technologies requires additional legal safeguards. AI tools make it possible to process personal data on a large scale and may sometimes use information belonging to special categories of personal data, commonly referred to as sensitive data.
Algorithms may reproduce existing biases
One of the most serious risks associated with the use of artificial intelligence in employment is its potential to perpetuate unequal treatment. AI models are trained on large datasets that may reflect historical biases and inequalities in the labour market.
For example, if historical data show that members of a particular group were more frequently recruited for certain positions, an AI system may interpret this relationship as a pattern that should be repeated. As a result, applicants may be assessed not only on the basis of their qualifications, but also according to indirect characteristics associated with their gender, age, ethnic background, disability or other legally protected attributes.
Poland’s Labour Code requires equal treatment of employees regardless of factors including gender, age, disability, race, religion, nationality, political views, trade union membership, ethnic origin, religious denomination and sexual orientation.
The President of UODO warns, however, that existing regulations may be insufficient in view of the technical capabilities of modern artificial intelligence systems.
Artificial intelligence may analyse too much data
Another concern is the scope of information used by algorithms. Under Article 22¹ of the Polish Labour Code, an employer may request only specified information from a job applicant, including their name, date of birth, contact details, education, professional qualifications and employment history, where such information is necessary for the role concerned.
AI tools may nevertheless draw conclusions from information that an employer should not consider during recruitment. This could include data obtained from social media, a candidate’s manner of speaking, behaviour during an online interview, facial expressions, tone of voice, place of residence or other apparently neutral information that may indirectly reveal protected personal characteristics.
In practice, a system could therefore make or recommend a decision on the basis of data extending beyond the statutory list of information that an employer is entitled to request from a job applicant.
Applicants should know why they were rejected
The President of UODO has also highlighted the right established under the General Data Protection Regulation not to be subject to a decision based solely on automated processing where that decision produces legal effects or similarly significantly affects an individual.
The rejection of an application by a recruitment system may have a substantial impact on a person’s professional and financial circumstances. Any national legislation permitting algorithms to make such decisions should therefore include appropriate safeguards.
A person assessed by an AI system should be able to understand how the tool operates, challenge its decision and obtain meaningful human intervention. It is also necessary to determine who is responsible for processing the data and making the final decision: the employer, the system provider or another entity involved in the recruitment process.
Recruitment AI systems are classified as high risk
The European Union’s Artificial Intelligence Act classifies systems used in employment, worker management and access to self-employment as high-risk AI systems.
This applies in particular to tools used to recruit or select candidates, publish targeted job advertisements, analyse and filter applications, and evaluate applicants during interviews. It also covers systems used to make decisions affecting employment conditions, promotions, termination of employment, task allocation, and the monitoring and evaluation of workers.
Under Article 86 of the AI Act, a person affected by a decision made on the basis of output from a high-risk AI system may request a clear and meaningful explanation of the role played by artificial intelligence in the decision-making process and of the main elements of the decision.
This is particularly important where a decision produces legal effects or similarly has a significant impact on an individual’s fundamental rights.
Assessments of the impact on privacy and fundamental rights are necessary
According to the President of UODO, any legislation permitting the use of AI systems in employment should be preceded by a data protection impact assessment.
Such an assessment would help determine what information will be processed, the purposes for which it will be used, how long it will be retained and what risks the use of algorithms may create for applicants and employees.
A data protection impact assessment should also help employers and technology providers demonstrate that their processing activities comply with the GDPR. It should establish whether the proposed arrangements contain adequate safeguards for the rights and freedoms of the people whose data are being analysed.
Separately, Article 27 of the AI Act requires an assessment of the impact of certain high-risk systems on fundamental rights, including the right to privacy. The President of UODO emphasises that the assessment required under artificial intelligence legislation should complement, rather than replace, the data protection impact assessment required under the GDPR.
Humans should retain oversight of algorithmic decisions
High-risk AI systems must comply with requirements concerning risk management, the quality of the data used, technical documentation, record-keeping, transparency, human oversight, accuracy and cybersecurity.
People affected by such systems should be aware that artificial intelligence is being used. Legislation should also clearly define the purposes for which AI tools may be deployed and identify the entity responsible for processing personal data.
Meaningful human oversight should remain a central safeguard. A person responsible for recruitment cannot simply approve an algorithm’s recommendation automatically. They must have the necessary knowledge, authority and competence to assess the output, modify it or reject it altogether.
UODO offers support in drafting new legislation
Mirosław Wróblewski has formally asked the Ministry of Family, Labour and Social Policy to begin legislative work aimed at providing applicants and employees with effective protection against discrimination resulting from the use of artificial intelligence.
The President of UODO has also offered the authority’s expert support in reviewing proposed measures for compliance with the GDPR.
The objective of any new regulations would be to reconcile employers’ ability to use modern technologies with the protection of privacy, the principle of equal treatment and the right of applicants to understand the genuine reasons behind decisions made during recruitment processes.
