SOUPS 2025 Poster Session

The computing world increasingly focuses on data collection, and the integration of advanced IT technologies creates new privacy and security vulnerabilities. Traditional approaches to security and privacy testing lack the scale and diversity needed to anticipate the full range of potential vulnerabilities. This ongoing work proposes using large language models (LLMs) to identify these vulnerabilities by having LLMs role-play personas of diverse users, including threat actors, regular users, and security practitioners. We created a pool of 128 individual personas derived from security and privacy literature and developed a framework to evaluate how effectively LLMs can embody these personas across standardized security scenarios. We validate persona simulation using this framework.

Driving a car is an important part of daily life for most people in the United States. However, people have a limited understanding of the privacy risks associated with data collected in an automotive context. The goal of this research was to investigate people’s awareness and perceptions of data collected about them by their cars, to better understand how their existing knowledge might affect how they think about driving data privacy. This poster presents preliminary findings from interviews focusing on participants' reactions to data collected about their driving over a 12 week period. Participants were surprised by their driving data when it did not match what they remembered about their driving, and when it revealed broader patterns about their lives beyond their cars. Surprise signals an expectation violation due to unexpected data practices, which indicates data collection and use that participants did not anticipate. People cannot make informed privacy decisions about data and inferences they are not aware of, and so identifying when and why surprise occurs can help privacy designers create interventions targeted towards gaps in user knowledge.

Digital platforms leverage user data to deliver personalized ads, yet users suspect that offline behaviors, spoken conversations, drive targeting. We surveyed 35 participants using mixed methods to explore beliefs about audio-based ad targeting. 63% of participants indicated that they thought audio-based ad targeting is used. We assessed whether participants would find other explanations for such targeting compelling, finding that technical/legal clarifications had limited impact, with distrust in voice assistants correlating strongly with belief persistence. Low trust in voice assistants and high concern about device listening correlated with these beliefs; staff reported greater worry than students. Our findings highlight enduring folk theories of ad targeting and the limits of technical correction.

DNS is susceptible to attacks such as cache poisoning, DDoS amplification, and privacy leakage. Despite the availability of defenses like DNSSEC, QNAME minimization, and source‐port randomization, adoption has remained surprisingly low. This study investigates the persistent gap between recommended DNS security practices and their practical implementation by examining how human factors and organizational constraints influence resolver operators' security decisions across diverse operational contexts. Through semi-structured interviews with resolver operators from diverse organizations, we identify key usability and operational barriers: configuration complexity that increases the risk of errors, performance-security tradeoffs that lack clear guidance, limited threat visibility that diminishes urgency, and the absence of contextual decision support. Operators frequently prioritize availability over proactive defenses and navigate organizational structures with distributed responsibilities and limited coordination. These factors contribute to the persistent under-deployment of known security mechanisms. Our findings inform the design of more usable DNS security tools and implementation strategies that better align with the constraints and mental models of operators.

Red team exercises evaluate an organization’s security by simulating attacks based on real-world adversary techniques. Red team exercises typically involve three stakeholders: the red team that conducts simulated attacks, the blue team that defends against them, and the coordination team that oversees the exercise. While several existing studies explored technical approaches to increase the efficiency of red team exercises, such as automating simulated attacks, the challenges faced by each stakeholder during the exercise are not well-studied. This study explores factors hindering the smooth execution of red team exercises through interviews with involved security professionals. In addition to the red team, the interview also targets coordination team members. As a result, we identified not only technical issues (e.g., difficulty in automating tasks), but also new non-technical challenges such as motivation gaps between stakeholders and a lack of adversarial skills and knowledge.

Virtual reality (VR) applications collect extensive user data, often without users’ full awareness. Current privacy interfaces in VR are frequently adapted from 2D systems and fail to leverage the immersive, spatial nature of VR. In this work, we explore how game elements can be used to design more engaging and effective privacy interactions in VR. Using a VR escape room environment, we conducted in-VR sketching sessions with novice game designers (n=12). Participants developed 17 privacy interaction concepts, 4 of which are highlighted in this poster for their creative use of game mechanics and metaphors. Our findings suggest that integrating privacy interactions into the game environment can raise user awareness and interest, but also highlight risks—such as blurring the line between gameplay and real privacy choices. We discuss these tensions and propose directions for refining and evaluating gamified privacy interfaces in VR.

Much biometric data collection is necessary to ensure comfortable game play in virtual reality (VR). On Oculus devices, the most popular consumer VR headsets, application access to user biometric data is moderated using permissions, similar to the Android permissions system.

We seek to understand if the current Oculus permissions framework effectively allows users to understand what data is being collected about them by various applications in VR. Through an interview study with 25 participants, we assess users' understanding of VR, permissions pop-ups in VR, and data collection practices in various VR applications. We seek to identify features guiding users' mental models of data collection in immersive modalities.

We explore how TikTok content supports individuals facing cyberbullying by analyzing the types of advice shared online. We focus on two key areas: (1) the types of advice provided, such as coping strategies, reporting mechanisms, and peer-driven support and (2) how this informal, community-generated guidance compares to expert recommendations from psychologists and cyber-safety organizations. Using qualitative thematic analysis, we contribute to the understanding of the support strategies shared on TikTok and evaluating its alignment with established expert guidelines.

To identify which design elements have the greatest impact on password memorability, we analyze 103 papers culled from 14,354 across 10 or more years from 14 conferences, personal bookmarks, related work sections, and keyword searches. We identify six key factors which we intend to study further using factor analysis in a user study.

A Large Language Model (LLM) powered GUI agent is a specialized autonomous system that performs tasks on the user's behalf according to high-level instructions. It does so by perceiving and interpreting the graphical user interfaces (GUIs) of relevant apps, often visually, inferring necessary sequences of actions, and then interacting with GUIs by executing the actions such as clicking, typing, and tapping. To complete real-world tasks, such as filling forms or booking services, GUI agents often need to process and act on sensitive user data. However, this autonomy introduces new privacy and security risks. Adversaries can inject malicious content into the GUIs that alters agent behaviors or induces unintended disclosures of private information. These attacks often exploit the discrepancy between visual saliency for agents and human users, or the agent's limited ability to detect violations of contextual integrity in task automation. In this paper, we characterized six types of such attacks, and conducted an experimental study to test these attacks with six state-of-the-art GUI agents, 234 adversarial webpages, and 39 human participants. Our findings suggest that GUI agents are highly vulnerable, particularly to contextually embedded threats. Moreover, human users are also susceptible to many of these attacks, indicating that simple human oversight may not reliably prevent failures. This misalignment highlights the need for privacy-aware agent design. We propose practical defense strategies to inform the development of safer and more reliable GUI agents.

Phishing affects everyone, but certain groups are at a higher risk of being phished than others. Prior research has pointed to language skills impacting phishing detection. We conducted a survey with the aim of understanding English as a Second Language (ESL) speakers' experiences with phishing compared to native English speakers to find out if one group was at a greater risk of falling for phishing than the other. We surveyed 100 native Mandarin, Korean, Spanish, and English speakers, 400 participants in total, who are living in the United States and are fluent in English. We presented the participants with phishing, ambiguous, and legitimate email simulations. We found that ESL speakers are at a lower risk of phishing than native English speakers. Additionally, we identified three key parts of the ESL experience of phishing, covering the weariness of receiving emails in their native language, the importance of familiarity, and the consideration of language specific nuances. Our findings show that just because someone is ESL does not mean they are necessarily more at risk of being phished. In the case of ESL speakers in the United States who are fluent, the opposite is true, this group is less likely to be phished when compared to Native English speakers.

As AI assistants like ChatGPT and Google Gemini become increasingly embedded in academic and everyday contexts, some universities have introduced institutional tools to address privacy and data security concerns. To examine how trust, usability, and privacy perceptions influence tool choice, we conducted a quantitative survey of 260 University of Michigan students, staff, and faculty. The survey collected data on usage patterns, perceived value, and user concerns, with additional open-text responses providing additional context.

Results show that while commercial AI tools are preferred for their accuracy and efficiency, university-developed tools are rated higher on ethical standards, transparency, and data privacy. Paid commercial tools like ChatGPT Plus were rated significantly higher in user satisfaction and performance (p = 0.00089, paired t-test). These findings suggest that institutional tools could improve adoption by enhancing usability, while commercial tools may benefit from greater transparency and privacy safeguards.

This study examines the current state of the passkey user experience across various websites, aiming to determine how well these passkey deployments align with the design guidelines recommended by the FIDO Alliance. We gathered information from 111 different websites between January and May 2025, examining what it is like for users to find, set up, use, and remove passkeys. We found that the passkey user experience varies significantly across the websites in our dataset. However, through our analysis, we observed similar passkey design patterns implemented amongst clusters of websites. We also observed several problematic, consistent design patterns implemented across several websites that create usability and security challenges for users. Our results offer usability and security recommendations for passkey implementers and advocate for the improvement of some design patterns in the FIDO Alliance Design Guidelines.

As wearable activity trackers (WATs) gain popularity in the U.S., concerns about their under-regulation under current privacy laws are increasingly pertinent. This study seeks to align efforts to address these gaps with user expectations by exploring how WAT users perceive privacy risks and regulatory gaps. Based on semi-structured interviews with 16 U.S. users, we find that participants identify 3 key risks: opaque data practices, location exposure, and unauthorized disclosure of sensitive health conditions. Users also express strong support for regulatory safeguards to address them. Users specifically call for enhanced transparency, greater control over data, enforceable restrictions on data sharing, and child-specific protections. Despite recognizing privacy risks, most users remain open to data sharing to trusted brands or for research purposes, suggesting that new laws need not curb WAT data sharing. We conclude by offering user-centered policy recommendations, including expanding public education on available protections and integrating new safeguards into HIPAA, given its high recognition among users.

This study investigates how the Hot Hand Fallacy, a well-established cognitive bias, influences decision-making in cyber attack contexts. Specifically, through an online experiment, we examined whether prior success with a particular brute force technique biases attackers toward repeating that technique, even when such success is unrelated to future outcomes. Seventy-five participants with security knowledge completed two main sets of questionnaires: the Cyber Method (CM), featuring attack scenarios designed to test the hot hand effect, and the Established Method (EM), based on canonical hot hand fallacy questionnaires. In the CM portion, participants assumed the role of a cyber attacker, reviewed NMAP scan reports, and selected among four brute force sub-techniques across three experimental conditions: Control, Moderate Streak, and Extended Streak. Conditions varied in the amount of prior success information provided. Results show that participants were significantly more likely to repeat the same attack after a success compared to a failure across all conditions. Such findings can contribute to understanding how cognitive biases such as the hot hand fallacy may shape attacker behavior, offering insights for cybersecurity defense strategies.

Generative AI risks such as bias and lack of representation impact people who do not interact directly with GAI systems, but whose content does: indirect users. Several approaches to mitigating harms to indirect users have been described, but most require top-down or external intervention. An emerging strategy, prompt injections, provides an empowering alternative: indirect users can mitigate harm against them, from within their own content. Our approach proposes prompt injections not as a malicious attack vector, but as a tool for content/image owner resistance. In this poster, we demonstrate one case study of prompt injections for empowering an indirect user, by retaining an image owner’s gender and disabled identity when an image is described by GAI.