As generative AI chatbots such as ChatGPT become increasingly embedded in everyday life, they are also being used for sensitive and personal purposes, including seeking information about sexual and reproductive health (SRH). At the same time, growing restrictions on women’s reproductive rights in the United States and globally have heightened concerns about how personal data might be tracked, shared, or exposed.
New research carried out by STAI CDT aligned PhD student Ina Kaleva and her co-authors, examines this intersection of technology, privacy, and women’s rights by exploring how women in the U.S. use generative AI chatbots to ask intimate questions about SRH.
Highlighting the risks of intimate surveillance
Ina and her co-authors’ research draws attention to the issue of intimate surveillance (i.e. the tracking, use, or exposure of highly sensitive personal data, especially data related to sexual and reproductive health). As Ina explains, “this is a serious concern for both privacy and women’s rights”.
Why user voices matter
Ina and her co-author’s research is based on in-depth interviews with women who have used generative AI chatbots for intimate health-related questions. This approach reflects a belief of the researchers that innovation should be driven by ethical principles and that people should be actively involved in shaping the technologies they use.
For Ina this means, “actively listening to users, supporting them in expressing their views, taking those views into account, involving them in decision-making, and empowering them”.
Affected communities and wider implications
The issue of intimate surveillance affects a wide range of people, as it includes both users of generative AI chatbots and non-users, since someone can intentionally or unintentionally share another person’s sensitive information without their consent.
More broadly, the way these systems handle sensitive data has implications for privacy, trust, and safety at multiple levels. It impacts women, technology developers, policymakers, clinicians, and society as a whole.
Design and policy impact
This important research has both design and policy implications for protecting users’ privacy in generative AI chatbots including educational campaigns for both users and clinicians, integrating privacy within the design and development of GenAI chatbots, interactive privacy features, greater transparency (e.g., explainable AI), and regulation reforms such as co-regulation between the industry and the regulatory agencies.
While prioritising safety and privacy may slow the pace of innovation, the researchers argue that rapid technological development should not come at the expense of ethical and safe implementation.
To balance this, Ina and her fellow researchers are advocating for a co‑regulatory approach, involving collaboration between industry and regulatory bodies, rather than relying solely on government regulation or industry self‑regulation.
Key findings and emerging questions
One of the most prominent findings from the study is users’ fear of criminalisation. Many women, particularly those living in states with more restrictive abortion laws, expressed concern that abortion‑related conversations could be monitored, flagged, or otherwise exposed through their interactions with AI chatbots.
The research also revealed tensions between user expectations and industry practices. While participants were uncomfortable with the idea of their data being used for profit, for example through targeted advertising, some AI providers have signalled that advertising may be a potential use of chatbot interaction data.
For Ina, one of the most interesting and compelling aspects of this research has been the new set of important questions raised by participants themselves, or inspired by their insights, which open important avenues for future work.
For example:
- Are certain outputs, such as those related to abortion, censored across different locations?
- How might user data or public data, like polarised social media content (e.g., opinions on abortion), influence GenAI responses?
- Are abortion-related conversations classified as “illegal activities” under moderation policies?
- How do contradicting laws such as anti-abortion laws and privacy laws interact with each other?
Why this matters beyond academia
Although this research focuses on women’s privacy and generative AI, its relevance extends across disciplines and sectors. It speaks to the broader principles of responsible research and innovation (RRI), emphasising the importance of designing products and services that are centred around users’ needs and concerns.
This is particularly potent given the increasingly widespread use of Generative AI tools. As Ina explains, “even if someone is not directly affected within their professional field, it is highly likely that they or someone they know, interact with these systems in everyday life”.
More information:
Ina and her co-authors presented the paper, ‘Privacy and Safety Experiences and Concerns of U.S. Women Using Generative AI for Seeking SRH Information’ at the ACM (Association for Computing Machinery) CHI conference on Human Factors in Computing Systems (CHI 2026).
Header Image:
Jamillah Knowles & Digit / https://betterimagesofai.org / https://creativecommons.org/licenses/by/4.0/