Welcome to the 2023-24 DINS Seminar Series!
Please note that this page will be updated as more information becomes available, so check back often!
Next Event:
Friday, March 29, 2024
Practical Mobile Sensing at Scale
Pengfei Zhou, Assistant Professor, Department of Informatics and Networked Systems, School of Computing and Information
(in cooperation with the Telecommunications Seminar)
1:00 pm to 2:00 pm
Room 411 IS Building, 135 North Bellefield Avenue, Pittsburgh, PA 15213
Abstract: Enabling large-scale sensing with mobile devices is crucial for various human-centered applications. However, in practice, environmental dynamics and the lack of well-curated sensor data make achieving large-scale accurate sensing across different scenarios extremely challenging. In this presentation, I will discuss our recent research progress on addressing these challenges using Inertial Measurement Unit (IMU) sensors on mobile devices. Specifically, I will introduce a foundational model designed to facilitate various large-scale sensing models using advanced self-supervised techniques inspired by BERT. I will then delve into our proposed universal Human Activity Recognition (HAR) framework, which aims to tackle data heterogeneity in real-world scenarios. Our approach involves augmenting data with the physics of the IMU sensing process and employing novel data augmentation techniques to leverage both unlabeled and labeled data effectively. Through extensive experiments across four open datasets, we demonstrate the superior performance of our approach in adapting HAR models to diverse environments.
Pengfei Zhou is an Assistant Professor in the Department of Informatics and Networked Systems, School of Computing and Information at the University of Pittsburgh. Prior to joining Pitt, he was a Research Scientist at Advanced Digital Sciences Center (ADSC), Illinois at Singapore, a Research Fellow at Alibaba-NTU Joint Research Institute (JRI) and a Technical Advisor for Alibaba Group. He founded fayfans Co., Ltd developing AIoT products in 2016. He received his B.E. degree in the Department of Automation from Tsinghua University, and his Ph.D. degree in the School of Computer Science and Engineering from Nanyang Technological University.
More Events To Come!
Previous Events:
Friday, March 1, 2024
Profiling humans from their voice: A journey from the past to future generation technologies
Rita Singh, School of Computer Science, Carnegie Mellon University
12 noon to 1 pm
Conference Room (Fifth Floor) in 130 N. Bellefield Building, across the street from the IS Building
In this talk, we will explore the fascinating domain of human voice profiling, a field at the confluence of voice forensics and artificial intelligence (AI) that I have been developing since 2014. The human voice serves as a unique and dynamic biometric signature, rich in personal information and responsive to a variety of influencing factors. This presentation will chart the evolution of voice profiling techniques—from traditional signal processing to advanced AI-driven methods—highlighting the key methodologies used to delve deeper and deeper into the human persona through voice. These methodologies essentially emerge by connecting the dots between a wide range of areas—from physics and biomechanics to machine learning and AI, to cytogenetics and genomics. As we navigate through the complexities and mysteries of the human voice, as a biometric identifier and a medium to infer the speaker's state, we uncover its profound implications for future human-machine interactions. As the world progresses towards Artificial General Intelligence (AGI) —we also speak of its physical embodiments. We will discuss why the science of human profiling is expected to play a pivotal role in embodied AGI systems of the future.
Rita Singh is an Associate Research Professor at the CMU’s School of Computer Science/Language Technologies Institute, with affiliations to three other departments. At CMU, she leads the Center for Voice Intelligence and Security (CVIS: http:// cvis.cs.cmu.edu/), and co-leads the Machine Learning for Signal Processing and Robust Speech Processing research groups. She has worked on speech and audio processing for over two decades. Since 2014, her work has been focused on developing the science of profiling humans from their voice, a niche area at the intersection of Artificial Intelligence and Voice Forensics. The technology pioneered by her group has led to three world firsts: In 2018, her team created the world’s first voice-based profiling system, demonstrated live at the World Economic Forum. In 2019 her group also created the world’s first instance of human voice – that of the artist Rembrandt – generated based on evidence from facial images. In 2020, her team conceptualized and enabled the first voice-based detection system for Covid-19. She is the author of the book “Profiling Humans from their Voice,” published by Springer-Nature in 2019. She has assisted multiple international agencies in analyzing voice evidence for identifying and profiling potential suspects in crimes under investigation. Her contributions have been recognized in global media, with several hundred mentions in various national and regional newspapers, magazines, online articles, TV and radio programs, podcasts and private talks, including a few church sermons.
Friday, February 23, 2024
Formal Methods or Usable Security: Why Not Both?
McKenna McCall, Postdoctoral Researcher in the Software and Societal Systems Department, Carnegie Mellon University
11:00 am to 12 noon
Room 305 Information Science Building, 135 North Bellefield Avenue, Pittsburgh, PA 15213
Abstract: Formal methods research involves using mathematical techniques to specify and verify properties of software and hardware systems. In security and privacy research, formal methods can lead to strong, provable security guarantees—and typically leave questions about how humans might interact with these systems unanswered. Indeed, formal methods and usable security are traditionally distinct areas of research. In this talk, I will demonstrate how techniques from both research areas can be applied—or even combined—to create solutions that are simultaneously mathematically rigorous and usable. In one project, we revisit static analysis tools for home IoT users from a usable security lens and investigate the usability and utility of the workflow involved in using the tools. Later in the talk, I will describe a project with a formal methods focus where we propose a new technique for preventing undesirable information flows on the web. We argue that this approach is usable in more realistic scenarios than what is proposed by prior work—without sacrificing security.
McKenna McCall is a postdoctoral researcher in the Software and Societal Systems Department at Carnegie Mellon University supervised by Lorrie Cranor and Lujo Bauer. She received her PhD from Carnegie Mellon University in 2023, advised by Limin Jia. McKenna’s research spans fields from information flow control and programming languages to security and privacy for home IoT and confidential computing. She is particularly interested in research where formal methods and usable security intersect, and combines techniques from both research areas to produce results that incorporate mathematical rigor as well as usability.
November 16, 2023
Pathways for Diversifying and Enhancing Science Advocacy
Fernando Tormos-Aponte, Assistant Professor, Department of Sociology, University of Pittsburgh
11:00 am to 12 noon
Conference Room, Fifth Floor, 130 North Bellefield Building (across the street from the IS Building)
Science is under attack and scientists are becoming more involved in efforts to defend it. The rise in science advocacy raises important questions regarding how science mobilization can both defend science and promote its use for the public good while also including the communities that benefit from science. This article begins with a discussion of the relevance of science advocacy. It then reviews research pointing to how scientists can sustain, diversify, and increase the political impact of their mobilization. Scientists, we argue, can build and maintain politically impactful coalitions by engaging with and addressing social group differences and diversity instead of suppressing them. The article concludes with a reflection on how the study of science-related mobilization would benefit from further research.
November 2, 2023
Personalization in the Age of Cyber-Physical-Social Systems
Bereket Yilma, Lecturer, Department of Computer Science, and Researcher, Computational Interaction (COIN) Group, University of Luxembourg
Nowadays, personalization has garnered significant traction in the context of smart environments collectively termed Cyber-Physical-Social System (CPSS). CPSS encompasses physical spaces such as smart cities, smart homes, schools, offices, museums, and factories where humans and sensor-enabled “intelligent devices” coexist. As these environments are continuously evolving in complexity, personalization presents both a promise and a challenge. In this talk, I discuss practices for designing personalization in dynamic interactive environments. Specifically, I focus on both scenarios that require knowing where to apply pure personalization and ones that necessitate departing from a sole optimization of user satisfaction to bring multi-stakeholder awareness. I showcase my recent research leveraging computational methods and machine learning algorithms to demonstrate how the challenges of implementing personalization in dynamic CPSS environments can be tackled. Particularly, I present a smart museum scenario which features a case study from the National Gallery in London. I discuss our findings from a small-scale and a large-scale user-centric evaluation highlighting how this research contributes to our understanding of delivering personalized content in dynamic environments, its implications and explain how it can benefit other domains where personalization has gained momentum.
Dr. Bereket Yilma is a Computer Scientist specializing in optimization and applied Machine Learning. He holds a PhD in Automatic Signal and Image Processing, and Computer Engineering. Currently, he works as a researcher within Computational Interaction (COIN) group at the University of Luxembourg and is a Lecturer in the Department of Computer Science. He also serves as the instructor of Recommender Systems in the Doctoral School of Computer Science and Computer Engineering. His research focuses on various aspects of Human-Centered Artificial Intelligence, including Recommender Systems, Adaptive User Interfaces, and Personalization in the context of Smart Interactive Environments aka Cyber-Physical-Social Systems (CPSS). He also actively contributes to Brain-Computer Interfaces (BCI) research within the framework of the BANANA project, Brainsourcing for Affective Attention Estimation. As an Associate Chair of SIGCHI, Dr. Yilma plays a key role in the HCI community. He is an Instructor at the SIGCHI CIX Summer Schools and serves as a PC member and reviewer for flagship HCI/ML venues, and ACM Conferences, such as CIKM, IUI, SIGIR, AAAI, LOD, ICML, ICLR, and NeurIPS.
October 19, 2023
Safe and versatile human-robot collaboration
Changliu Liu, Assistant Professor, Robotics Institute, School of Computer Science, Carnegie Mellon University
Abstract: Human-robot collaboration (HRC) is one key component to achieving flexible manufacturing to meet the different needs of customers. This talk will cover some of our recent work towards safe and versatile human-robot collaboration in production lines. To ensure safety proactively, we introduced a method to predict how the robot’s behavior would affect the human’s behavior; and then let the robot choose actions either to show courtesy to the human or to influence the human toward system optima. To make the collaboration versatile, we mainly focus on enabling efficient communication between the human and the robot though both novel hardware design and novel algorithmic design. We introduced a low cost textile-based tactile skin which is able to detect gestures by human users. On the algorithmic side, we developed a one-shot learning from demonstration algorithm to efficiently translate human commends and a conditional collaborative handling process to make the robot quickly adapt to the preference of different users. We are developing a human-robot collaborative Lego assembly testbed (with a digital twin), which we hope to integrate all these previous work and then open source and share with the community as a potential benchmark system.
Bio: Dr. Changliu Liu is an assistant professor in the Robotics Institute, School of Computer Science, Carnegie Mellon University (CMU), where she leads the Intelligent Control Lab. Prior to joining CMU, Dr. Liu was a postdoc at Stanford Intelligent Systems Laboratory. She received her Ph.D. in Engineering together with Master degrees in Engineering and Mathematics from University of California at Berkeley and her bachelor degrees in Engineering and Economics from Tsinghua University. Her research interests lie in the design and verification of intelligent systems with applications to manufacturing and transportation. She published the book “Designing robot behavior in human-robot interactions” with CRC Press in 2019; and the book “Algorithms for verifying deep neural networks” in Foundations and Trends in Optimization in 2021. She is the founder of the International Neural Network Verification Competition launched in 2020. In 2022, she demonstrated their human-robot handover system to President Biden. Her work has been covered by IEEE Spectrum, ATI news, Robotiq Blog, etc; and has been recognized by NSF Career Award, Amazon Research Award, Ford URP Award, and many best/outstanding paper awards.
October 11, 2023
The Power of Beauty in Informal Learning
Maria Harrington, Associate Professor of Digital Media, in the Games and Interactive Media Program, Nicholson School of Communication and Media at the University of Central Florida
The Virtual UCF Arboretum is an immersive embodied experience similar to a real field trip used to understand human interconnectedness with nature, emotions, informal learning, beauty, perceptio
n, and shifts in attitudes. Dr. Maria Harrington, Associate Professor of Digital Media at the University of Central Florida, leads research projects focused on methods of modeling and visualizing natural environments and their use in immersive informal learning application design, development, and evaluation. She uses methods of geospatial data visualizations in Unreal Engine to create immersive embodied experiences using augmented and virtual reality. Her research studies investigate informal learning, human emotional and behavioral responses, and impacts on shifts in attitudes. Her research is transforming the way people perceive and understand the natural world.
October 5, 2023
Expanding digital pasts for the future: from data preservation to dynamic collaboration in archaeology
Elizabeth Arkush, Professor of Anthropology, University of Pittsburgh, and Alexander Martin, Associate Director of the Center for Comparative Archaeology
Archaeology is grappling with a deluge of data from field studies, including digitized and born-digital data, geospatial and relational data, and imagery (LiDAR, drone photomosaics, etc). Increasingly, archaeologists aim to synthesize this growing knowledge base from multiple sites and time periods into composite, comparative histories that can speak to contemporary questions: the deep human history of population growth, environment, and sustainable or unsustainable land use; long-term phases of warfare and peace across continents; measuring inequality or prosperity in past societies; and so on. Meanwhile, because archaeological methodologies are often destructive, there is an ethical imperative to preserve the data from our human heritage for the future in robust and long-lasting formats, and make it accessible to diverse users and publics. For over 10 years, the Comparative Archaeology Database at Pitt (https://www.cadb.pitt.edu/) has served as an open-access, bilingual digital repository for primary archaeological data and full metadata in easy-to-use formats. Here, we envision how to move towards new data architectures for archaeology that are collaborative and dynamic, accommodating the "data deluge" while maintaining responsible preservation.
Elizabeth Arkush is Professor of Anthropology at the University of Pittsburgh and Director of the Comparative Center for Archaeology. She is an archaeologist whose field research in the Peruvian Andes has focused on warfare, politics, landscape and climate in late pre-Columbian times. Her research relies on geospatial technologies and digital recording methodologies such as drone mapping, GIS analysis, and satellite image processing. As Director of the CCA, she manages programs to preserve and publish open-access archaeological data and to engage undergraduates in hands-on archaeology research. 
SEPTEMBER 21, 2023
Co-sponsored by Pitt Cyber
Why social media is so uniquely toxic to our mental health
Kristina Lerman, Principal Scientist, University of Southern California Information Sciences Institute
There is growing consensus that social media has created a mental health crisis among youth, with severe consequences to their emotional well-being, social functioning, and physical health. This crisis disproportionately affects girls and young women, widening gender gaps and hindering progress towards gender equality. However, the pathways of social media harm are not well understood. In this talk I present evidence for a potential mechanism, exploring it in the context of eating disorders, an often-fatal mental health condition that includes anorexia, bulimia and binge eating disorder. The rise in eating disorders has been linked to the proliferation of idealized body images on social media; however, the link between social media and eating disorders is more complex. I show how technological affordances of social media can create a “harm spiral” that traps people in toxic echo chambers that promote eating disorders. Specifically, social media makes harmful content easy to discover and its group dynamics encourage people to stay engaged, which exposes them to more harmful content that is deleterious to mental health. To address the youth mental health crisis, we urgently need to understand how social media harms mental health, and who is vulnerable to its detrimental effects.
Kristina Lerman is a Principal Scientist at the University of Southern California Information Sciences Institute and holds a joint appointment as a Research Professor in the USC Computer Science Department. Trained as a physicist, she now applies network analysis and machine learning to problems in computational social science, including crowdsourcing, social network and social media analysis. Her work on modeling and understanding cognitive biases in social networks has been covered by the Washington Post, Wall Street Journal, and MIT Tech Review. She is a fellow of the AAAI.