Jieshan Chen 陈洁珊

Research Scientist at CSIRO’s Data61
✉ Jieshan.Chen@data61.csiro.au · GitHub · Google Scholar · LinkedIn · Twitter

About me

Jieshan is currently a research scientist and UI intelligence team lead at CSIRO’s Data61, and the first TUM-IAS Dieter Schwarz fellow from 2024-2027. She received her Ph.D. degree in computer science at Australian National University, where she was advised by Zhenchang Xing and Chunyang Chen.

She works at the intersection of software engineering and human computer interaction. She is always passionate on addressing real-life problems that can actually impact people’s daily life. Her main research topics are around software usability and UI engineering. She is currently working on dark pattern detection, data visualisation, UI design search and generation, and mobile application usability and accessibility enhancement. Her works have been integrated into industrial tools (e.g., by Apple and Alibaba), benefiting billions of end-users.

Jieshan has published papers at top-tier CORE A* ranked conferences/journals, including ICSE, FSE, UIST, CHI, TOSEM and USENIX Security, and contanstly serves as PC members and reviewers for top-tier conferences and journal. She got one SIGSOFT Distinguished Paper Award at ICSE2020. See her Google Scholar Page for more details.

News

> 2023 - [2023-12] Our paper "Where is it? Tracing the Vulnerability-relevant Files from Vulnerability Reports" is accepted by ICSE'2024!
- [2023-12-05] Our paper "Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains" is accepted by TOSEM 2023!
- [2023-11] Invited as a PC Member for ICSE 2025
- [2023-08-05] Our paper "Unveiling the tricks: automated detection of dark pattern in mobile applications" is accepted by [UIST 2023](https://uist.acm.org/2023/)
- [2023-07] Invited as a PC Member for SANER 2024
- [2023-05] Invited as a PC Member for ASE 2023
> 2022 - [2022-12-01] Selected as a Junior PC Member at MSR 2023
- [2022-08-31] Gave a talk titled **"Understanding Fairness Issues in Machine Unlearning and Dark Pattern"** at [NIST Workshop on Mitigating AI Bias in Context](https://www.nccoe.nist.gov/get-involved/attend-events/workshop-mitigating-ai-bias-context)
- [2022-07-08] Released a preprint version of our work **"Extracting Replayable Interactions from Videos of Mobile App Usage"**
- [2022-01-10] Joined CSIRO's Data61 as a research scientist
- [2021-11-16] Our paper **"Towards Complete Icon Labeling in Mobile Applications"** accepted at [CHI 2022](https://chi2022.acm.org/)
- [2021-05-26] Presented our paper **"Wireframe-based UI Design Search through Image Autoencoder"** at ICSE 2021

Selected Publications

Please see full the publication list in Google Scholar

[WWW’25] 50 Shades of Deceptive Patterns: A Unified Taxonomy, Multimodal Detection, and Security Implications
Zewei Shi, Ruoxi Sun, Jieshan Chen, Jiamou Sun, Jason Xue, Yansong Gao, Feng Liu, Xingliang Yuan
To appear · Acceptance rate: 19.8% (409/2062) - Oral

[ICSE’25] DesignRepair: Dual-Stream Design Guideline-Aware Frontend Repair with Large Language Models
Mingyue Yuan, Jieshan Chen*, Zhenchang Xing, Aaron Quigley, Yuyu Luo, Gelareh Mohammadi, Qinghua Lu, Liming Zhu
Arxiv · GitHub · Acceptance rate: 21.3% (245/1150)

[UIST’24] GPTVoiceTasker: Advancing Multi-step Mobile Task Efficiency Through Dynamic Interface Exploration and Learning
Minh Duc Vu, Han Wang, Zhuang Li, Jieshan Chen, Shengdong Zhao, Zhenchang Xing, Chunyang Chen
ACM · Arxiv · GitHub · Acceptance rate: 24% (146/608)

[ICSE’24] Where is it? Tracing the Vulnerability-relevant Files from Vulnerability Reports
Jiamou Sun, Jieshan Chen, Zhenchang Xing, Qinghua Lu, Xiwei (Sherry) Xu, Liming Zhu
ACM · GitHub · Acceptance rate: 22.3% (234/1051)

[TOSEM’23] Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains
Yu Cheng, Jieshan Chen*, Qing Huang, Zhenchang Xing, Xiwei Xu, Qinghua Lu
Nov 2023 · ACM · ArXiv · Talk · Tool

[UIST’23] Unveiling the tricks: automated detection of dark pattern in mobile applications
Jieshan Chen, Jiamou Sun, Sidong Feng, Zhenchang Xing, Qinghua Lu and Chunyang Chen
ACM · ArXiv · Talk · Dataset · Code · Acceptance Rate 25% (121/483)

[ASE’23] Let’s Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain
Qing Huang, Zhenyu Wan, Zhenchang Xing, Changjing Wang, Jieshan Chen, Xiwei Xu, Qinghua Lu
IEEE · Acceptance Rate 20.3% (134/661)

[Preprint ‘23] Towards Real Smart Apps: Investigating Human-AI Interactions in Smartphone On-Device AI Apps
Jason Ching Yuen Siu, Jieshan Chen*, Yujin Huang, Zhenchang Xing, Chunyang Chen
July 2023 · ArXiv

[Preprint ‘22] Extracting Replayable Interactions from Videos of Mobile App Usage
Jieshan Chen, Amanda Swearngin, Jason Wu, Titus Barik, Jeffrey Nichols and Xiaoyi Zhang
July 2022 · PDF

[CHI’22] Towards Complete Icon Labeling in Mobile Applications
Jieshan Chen, Amanda Swearngin, Jason Wu, Titus Barik, Jeffrey Nichols and Xiaoyi Zhang.
PDF · Talk · Acceptance Rate 12.5% (324/2,597) · Available in iOS

[ESEC/FSE’20] Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?
Jieshan Chen, Mulong Xie, Zhenchang Xing, Chunyang Chen, Xiwei Xu, Liming Zhu and Guoqiang Li
PDF · Tool Demo · Talk · Code · Acceptance Rate 28% (101/360) · Integrated into an industrial testing tool for daily testing apps of 1B+ users

[TOSEM’20] Wireframe-based UI Design Search through Image Autoencoder.
Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xin Xia, Liming Zhu, John Grundy, and Jinshui Wang.
PDF · Talk · Code

[ICSE’20] Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learning.
Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xiwei Xu, Liming Zhu, Guoqiang Li, and Jinshui Wang.
🏆 ACM SIGSOFT Distinguished Paper Award
PDF · Talk · Code · Acceptance Rate 20.9% (129/617)

* means corresponding authors.

Service

Organising Committee
· ACM Conference on Computer and Communications Security (CCS’24 – LAMPS Workshop, Program Co-Chair)

Program Committee
· IEEE/ACM International Conference on Automated Software Engineering (ASE’23, ASE’24-Artifact Evaluation), ASE’25
· ACM international conference on AI Foundation Models and Software Engineering (FORGE’25-Data and Benchmarking)
· International Conference on Software Engineering (ICSE’25, ICSE’26)
· ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’24)
· IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER’24-ERA, SANER’25-ERA)
· ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2023-SRC)
· ACM international joint conference on pervasive and ubiquitous computing (UbiComp/ISWC’23 Posters & Demos)
· IEEE International Symposium on Software Reliability Engineering (ISSRE’22-Industry, ISSRE’23-Industry Track)
· The Mining Software Repositories (MSR’23-Junior PC, MSR’22-Shadow PC)
· International Conference on Software and System Processes (ICSSP’23)
· ACM CHI Conference on Human Factors in Computing Systems (CHI’22-Computational UI Workshop)

Reviewer
· AACM Transactions on Computer-Human Interaction (TOCHI’24)
· ACM Symposium on User Interface Software and Technology (UIST’22, UIST’24)
· ACM CHI Conference on Human Factors in Computing Systems (CHI’22, CHI’23, CHI’24, CHI’25)
· ACM SIGCHI Conference on Computer-Supported Cooperative Work & Social Computing (CSCW’24)
· International Conference on Software Engineering (ICSE’24)
· IEEE Transactions on Software Engineering and Methodology (TOSEM’24)
· IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR’23)
· ACM international joint conference on pervasive and ubiquitous computing (UbiComp’22)
· International Journal of Human-Computer Interaction (IJHCI’22)
· IEEE International Symposium on Software Reliability Engineering (ISSRE’22)
· IEEE Transactions on Software Engineering (TSE)
· Information and Software Technology (IST Journal’21)
· The Mining Software Repositories (MSR’21)