publications

2024

1. IJCAI

   SaGol: Using MiniGPT-4 to Generate Alt Text for Improving Image Accessibility

   Yunseo Moon, Hyunmin Lee, SeungYoung Oh, and 1 more author

   In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Aug 2024

   Demo Track
2. Data in Brief

   Creating a bias-free dataset with food delivery app reviews under data poisoning attack

   Hyunmin Lee, SeungYoung Oh, JinHyun Han, and 1 more author

   Data in Brief, Aug 2024

   Abs

   In online food delivery apps, customers write reviews to reflect their experiences. However, certain restaurants use a “review event” strategy to solicit favorable reviews from customers and boost their revenue. Review event is a marketing strategy where a restaurant owner gives free services to customers in return for a promise to write a review. Nevertheless, current datasets of app reviews for food delivery services neglect this situation. Furthermore, there appears to be an absence of datasets with reviews written in Korean. To solve this gap, this paper presents a dataset that contains reviews obtained from restaurants on a Korean app which use a review event strategy. A total of 128,668 reviews were gathered from 136 restaurants through crawling reviews using the Selenium library in Python. The dataset consists of detailed information of each review which contains information about ordered dishes, each review’s written time, whether the food image is included in the review or not, and various star ratings such as total, taste, quantity, and delivery ratings. This dataset supports an innovative process of preparing AI training data for achieving fairness AI by proposing a bias-free dataset of food delivery app reviews with data poisoning attacks as an example. Additionally, the dataset is beneficial for researchers who are examining review events or analyzing the sentiment of food delivery app reviews.

2023

1. CSCW

   Visualizing the Carbon Intensity of Machine Learning Inference for Image Analysis on TensorFlow Hub

   Taewon Yoo, Hyunmin Lee, SeungYoung Oh, and 2 more authors

   In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing, Aug 2023

   Abs HTML PDF

   The increasing performance of machine learning (ML) models necessitates greater computing resources, contributing to rising carbon intensity in ML computing and raising concerns about computational equity. Previous studies focused on developing tools that enable model developers to view the carbon intensity of the ML models in the training process. Still, little is known about how to support ML developers in online communities to explore the carbon intensity of ML models during inference. We developed MIEV, a model inference emission visualizer, that supports ML developers on TensorFlow Hub to explore the carbon intensity of image domain models during the model Inference phase. We also provide insights into designing technologies that promote collaborative work among ML developers to drive sustainable AI development processes. To the best of our knowledge, this is the first attempt to interactively visualize the carbon intensity of ML models in online communities during the Inference phase.

2. IUI

   WATAA: Web Alternative Text Authoring Assistant for Improving Web Content Accessibility

   Hyeonhak Jeong, Minki Chun, Hyunmin Lee, and 2 more authors

   In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Aug 2023

   Abs HTML PDF

   Alternative (alt) text is essential for people with visual impairments to acquire information about image content through a screen reader. However, collecting images and creating alt text requires time and effort. To deal with this problem, automatically collecting images that have no alt text is essential to ensure that all image content in a web page has their alt text. Additionally, automatic alt text has limitations in accuracy and quality compared to human-created alt text despite the improvements in image recognition and natural language process technology. We present WATAA, a web alt text authoring assistant that collects images containing no alt text and suggests automatic alt text to help human alt text authors improve a web page’s accessibility.

3. IUI

   PORDE: Explaining Data Poisoning Attacks Through Visual Analytics with Food Delivery App Reviews

   Hyunmin Lee, Minki Chun, and Hyunggu Jung

   In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, Aug 2023

   Abs HTML PDF

   Artificial intelligence (AI) gives many benefits to our lives. However, biased AI models created by receiving data poisoning attacks may induce social problems. Therefore, developers must consider carefully whether the training data received a poison attack when developing an AI model. Data visualization is one of the methods to facilitate the analysis of the data required for checking if the training data received a poisoning attack. However, prior studies did not visualize real-world AI training data. Restaurant reviews in delivery apps are one of the cases of a poisoned dataset. Restaurants hold review events on delivery apps to encourage customers to write a positive review in return for certain rewards, thereby creating reviews with bias. In this study, we propose POisoned Real-world Data Explainer (PORDE) that explains data poisoning attacks through visual analytics with food delivery app reviews. The findings of this study suggest implications for securing safe training data and developing less biased AI models.

2022

1. KSC

   System for Analyzing Bias of Data to Develop Artificial Intelligence Models: a Case Study of Food Delivery App Reviews

   Hyunmin Lee, and Hyunggu Jung

   In Proceedings of Korea Software Congress, Aug 2022

   HTML
2. IEEE Access

   Development of Korean Food Image Classification Model Using Public Food Image Dataset and Deep Learning Methods

   Minki Chun, Hyeonhak Jeong, Hyunmin Lee, and 2 more authors

   IEEE Access, Aug 2022

   Abs HTML PDF

   Food image classification is useful in diet management apps for personal health management. Various methods for classifying food images in a particular country have been proposed in multiple studies. However, knowledge of Korean food image classification is limited. The objective of this study was to train a classification model for Korean food images. To train the classification model, we collected Korean food images from the AI-Hub platform, a public food image dataset. The images were pre-processed and augmented for model training. The proposed model was evaluated in an experiment using a Korean food image dataset and effectively classified Korean food images based on transfer learning. The findings of the study revealed that the model’s performance in classifying food images depended on the type of food. The findings have implications for the classification model training process using CNN and the Korean public food image dataset. Future work is required to improve the performance of a classification model, especially as it pertains to its poor performance for some food image types.

3. KIISE

   Toward Fairness in Artificial Intelligence: Uncovering Food Delivery App Reviews with Poisoning Attacks

   Hyunmin Lee, Minki Chun, Hyeonhak Jeong, and 1 more author

   Communications of the Korean Institute of Information Scientists and Engineers, Aug 2022

   HTML
4. KCC

   Auto Labeling Restaurant Notices of Delivery Applications using Text Classification Model

   Hyunmin Lee, and Hyunggu Jung

   In Proceedings of Korea Computer Congress, Aug 2022

   HTML