NYU Center for Data Science

Meet new CDS Faculty Fellow Jonathan Colner, joining us this fall. Having recently received his PhD in Political Science at UC Davis, Colner focuses on leveraging data science to understand local political institutions. His research develops tools for measuring city councils' policy-making activities, using AI to analyze data from sources like meeting minutes and municipal codes.

“I’m excited to join NYU CDS and work at the intersection of data science and local politics,” said Colner. He looks forward to collaborating with experts on these new tools to understand local governments better.

Meet the Fellow: Jonathan Colner This entry is part of our Meet the Fellow blog series, which introduces and highlights Faculty Fellows who have recently joined CDS.

The Global AI Frontier Lab was officially launched on May 30, 2024, marking a significant collaboration between Korea and NYU. Spearheaded by AI pioneers Yann LeCun (CDS founding director) and Kyunghyun Cho (CDS Professor), this initiative is set to push the boundaries of AI research. The lab will be housed in NYU’s Brooklyn facilities, drawing top talent from around the globe.

Republic of Korea Minister of Science and Information and Communication Technology Lee Jong-ho and NYU President Linda G. Mills highlighted the importance of this partnership. “I am confident that this global partnership, steeped in scholarly excellence, will make a transformative contribution to the field of artificial intelligence,” said President Mills.

The lab aims to foster innovation and sustainable AI development.

Read more: https://www.nyu.edu/about/news-publications/news/2024/may/korea-and-nyu-establish-global-ai-frontier-lab.html

A midsummer’s data science research: CDS at the ICLR, Jacob Pfau and William Merrill make a surprising discovery about large language model reasoning, and more on July’s research feature: https://mailchi.mp/nyu/the-cds-monthly-research-feature-july24-edition

That’s a wrap! CDS celebrated the successful 4th annual NYU AI School, which introduced AI and machine learning to undergraduates from diverse backgrounds. Organized by CDS and the Machine Learning for Language (ML²) Lab, the program aimed to make AI education accessible to all students. With 61% of participants from non-computer science majors, the program offered tracks for both beginners and those with coding experience.

The curriculum covered exciting topics such as machine learning, computer vision, natural language processing, and practical AI applications. Engaging panel discussions on ‘Careers in AI’ and ‘First Steps in ML’ provided valuable insights and advice.

NYU AI School is democratizing AI education, making it accessible to students from a wide variety of fields.

NYU AI School Wrapped: Democratizing AI Education CDS’s 4th annual NYU AI School introduced AI to a diverse group of undergraduates, aiming to make AI education accessible to all students.

A new study from CDS introduces the S4MI (Self-Supervision and Semi-Supervision for Medical Imaging) pipeline, significantly reducing the need for manual labeling in medical image analysis. Led by CDS Assistant Professor and Faculty Fellow Jacopo Cirrone, the research highlights improved classification and segmentation tasks, outperforming fully-supervised methods while requiring 50% fewer labels. “Our semi-supervised approach for segmentation requires 50% fewer labels across all evaluated datasets,” Cirrone noted.

Developed with the Courant Institute and NYU Grossman School of Medicine, this interdisciplinary effort aims to streamline medical imaging and improve clinical outcomes. The project also benefited significantly from the contributions of three students from the Fall 2022 MS capstone research project at CDS: Luoyao Chen, Mei Chen, and Jinqian Pan. Their project, recognized as one of the best capstone projects of the semester, focused on developing and testing the semi-supervised learning algorithm at the core of the pipeline.

Semi- and Self-Supervised Learning Help Clinicians Minimize Manual Labeling in Medical Image… CDS study reduces manual labeling in medical imaging by leveraging self-supervised and semi-supervised learning.

We are very pleased to share that CDS PhD Student Jacob Pfau received a Fast Grant from OpenAI’s Superalignment project! This grant supports innovative approaches in AI alignment and safety.

Jacob's project focuses on scalable oversight. When a super-human model detects a flaw that humans can't spot, ordinary prompting won't help. His solution uses in-context learned probabilities to identify counterfactuals. By sampling inputs near these probabilities, his method efficiently uncovers critiques known to the model but not to humans.

This research promises to enhance our understanding of AI reliability and oversight.

Congratulations, Jacob!

CDS researchers have introduced a promising new method to reduce biased reasoning in AI language models. Recently departed CDS Junior Research Scientist Miles Turpin, CDS Associate Professor of Linguistics and Data Science Samuel R. Bowman, CDS Research Scientist Julian Michael, and others, detail their approach in the paper "Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought."

This method involves training models to generate unbiased reasoning, even when biased prompts are used. According to Turpin, this training could help improve the faithfulness of model explanations, enhancing trust in AI outputs.

Read more about their findings here:

New Research Finds Method to Reduce AI Language Models’ Biased Reasoning In a paper last year, recently departed CDS Junior Research Scientist Miles Turpin, CDS Associate Professor of Linguistics and Data…

CDS students had the unique opportunity to engage with CDS Professor of Data Science, Computer Science, and Engineering & Institute Professor of Tandon School of Engineering Claudio Silva during a special Pizza Chat event. Prof. Silva shed light on his groundbreaking work in large-volume fossil scanning, known as PaleoScan. The project aims to revolutionize the study of fossils by enabling high-resolution 3D scanning and analysis.

Students were captivated by the potential of this technology to uncover new insights into our planet's ancient past. The event fostered a lively discussion about the intersection of data science and paleontology, leaving attendees inspired by the endless possibilities in this exciting field.

CDS PhD Student Christopher Ick’s latest work addresses a significant challenge in urban sound identification and localization. His paper, “SpatialScaper: A Library to Simulate and Augment Soundscapes for Sound Event Localization and Detection in Realistic Rooms,” presented at ICASSP 2024, introduces a powerful new tool for simulating sound data for machine learning models.

“SpatialScaper allows us to generate vast amounts of labeled sound data without the need for extensive manual annotation,” Ick explained. The project, developed in collaboration with CDS Assistant Professor Brian McFee, is available on GitHub for community contributions. Explore more about this innovative library and its practical applications in assistive technology, audio production, and beyond.

Read the full article:

Simulating Soundscapes: New Tool Enhances Machine Learning Models for Audio Localization CDS PhD Student Christopher Ick’s SpatialScaper tool, presented at ICASSP 2024, enhances sound event localization for machine learning…

CDS Clinical Associate Professor of Data Science and Psychology Pascal Wallisch has won the 2024 Golden Dozen Award for excellence in undergraduate teaching. Each year, the NYU College of Arts and Science recognizes 12 full-time faculty members for their outstanding contributions in the classroom, and it is the first Golden Dozen award for data science.

This year, Pascal was nominated by both students and faculty, adding to his recognition as an exceptional educator at NYU, as he previously won the Golden Dozen award (in 2016) and the Teach/Tech award (in 2020). Wallisch helped to steer the burgeoning undergraduate degree program in Data Science during the Coronavirus and the subsequent scaling-up phase.

The CDS Capstone Project provides students an unparalleled opportunity to apply cutting-edge skills to real-world challenges. Just ask NYU Abu Dhabi Assistant Professor of Political Science Aaron Kaufman, who mentored the award-winning Fall 2023 Capstone project "Partisan Bias and the US Federal Court System" by Annabelle Huether, Mary Nwangwu, and Allison Redfern, all CDS MSDS students at the time.

As Asst. Prof. Kaufman put it: "The Capstone [Project] gives our students a chance to not only learn the state of the art but to actively advance it. These students' skills are so far on the cutting edge that when they apply for jobs, they're dramatically set apart from students of other institutions who only attended classes."

The team leveraged advanced NLP techniques to predict the partisan leanings and topics of lower federal court decisions based on opinion text. By aggregating predictions to the judge level, they examined how individual judges' partisanship and case topics have evolved. A valuable first attempt at extracting this data to discern trends in judicial partisanship over time.

More info: https://cds.nyu.edu/capstone-project/

CDS PhD alum, Zhouhan Chen, now a visionary in combating social media abuse, recently discussed his journey from a Twitter security intern to founding Safelink Network in a recent YouTube interview with Xccelerate. He delved into the creation of Information Tracer, a tool born during the pandemic to fight misinformation. Zhouhan's work now aids journalists, security analysts, and researchers globally, making an impact on issues from election integrity to online hate speech. Watch the full interview to learn about his journey and innovations.

Interview: https://www.youtube.com/watch?v=HdoKMTMcmyQ

Information Tracer: https://informationtracer.com/

CDS researchers took center stage at ICLR 2024 in Vienna, showcasing groundbreaking work in deep learning. CDS Professor of Computer Science and Data Science Kyunghyun Cho was one of the keynote speakers and also co-authored one of the 5 Outstanding Paper Awards.

Another of those 5 Outstanding Paper Awards was won by CDS PhD student Zahra Kadkhodaie's paper on generalization in diffusion models. Meanwhile, CDS PhD student Angelica Chen presented her Spotlight (top 5%) paper, co-authored with CDS Faculty Fellow Ravid Shwartz-Ziv and CDS Professor Kyunghyun Cho.

The hits continued: CDS PhD student William Merrill presented his paper “The Expressive Power of Transformers with Chain of Thought,” while CDS PhD student Vishakh Padmakumar explored content diversity in LLM writing. Finally, Tim G. J. Rudner presented his study on Bayesian Neural Network surrogates for optimization.

Learn more:

CDS at ICLR 2024: A Showcase of Cutting-Edge Research CDS members presented pioneering research at ICLR 2024

Meet our summer 2024 CURP (CDS Undergraduate Research Program) scholars, who joined us today!

For those who may not be familiar, CURP is a research mentorship program that provides talented undergraduate students with the opportunity to work in-depth with our renowned faculty alongside a community of their peers.

Join us in welcoming our current CURP scholars!

For more details on the program, please visit our website: https://cds.nyu.edu/curp/

Attention NYU Undergrads! It's that time of year again: the Summer Declaration Period is now open. If you are interested in becoming a data science major or minor, make your declaration by July 12. Don't miss your chance! Visit our website for more information: https://cds.nyu.edu/undergraduate-program/

Need something to read on your summer getaway? Look no further 😎 Kyunghyun Cho separates hype from reality in AI advancement, Claudia Skok-Gibbs unveils the “hidden code of life,” and more on this month’s CDS research feature: https://mailchi.mp/nyu/the-cds-monthly-research-feature-june24

The 'stochastic parrot' hypothesis famously questions whether AI can truly understand or merely mimics human language. Refuting this, new research by a team of researchers in CDS Associate Professor of Computer Science and Data Science Andrew Gordon Wilson’s research group, co-led by CDS PhD student Sanae Lotfi, offers robust mathematical proofs that large language models can indeed generalize beyond their training data.

The team also includes CDS PhD student Yilun Kuang, CDS Postdoc Researcher Micah Goldblum, CDS Instructor Tim G. J. Rudner, and CMU postdoc Marc Finzi.

Learn more:

Do Large Language Models Really Generalize? This Paper Says Yes Latest study by CDS’ Sanae Lotfi, Andrew Gordon Wilson, and others, proves LLMs’ ability to genuinely generalize.

CDS recently hosted a captivating data management seminar featuring talks by renowned researchers Marcelo Arenas and Leonid Libkin.

Arenas, a professor of computer science at the Institute for Mathematical and Computational Engineering at the Pontificia Universidad Católica de Chile, delved into "A data management approach to explainable AI," exploring the potential of logic as a declarative language for model interpretability.

Libkin, a professor of computer science at the University of Edinburgh and query language researcher at RelationalAI, presented "Graph Query Language (GQL) for researchers in a nutshell," offering insights into the newly standardized language. The engaging talks were followed by a lively discussion over wine at the 7th floor open space of CDS.

CDS Student Research Showcase Highlights Cutting-Edge Work

On Friday, April 26, CDS hosted a student research showcase featuring lightning talks from our talented student researchers. Komal Sharma discussed equity in climate change research, while Yuqing Cui presented on data-centered decision-making in a DA's office to balance public safety and incarceration. Keya Shukla spoke about video recommendation systems, and a team consisting of Diego Lopez, Zoe Hsu, Olive Song, and Zihang Xia explored network intrusion detection using deep learning.

A***n Jain, Aradhita Bhandari, Mallory Sico, and Gail Batutis presented their work on user-centric AI models for assisting the blind. Palak Bansal and Hoa Duong showcased their research on GANs for causal inference, harnessing conditional independence. Ryan Li rounded out the presentations with a talk on the economic impact of language model performance.

The event concluded with an opportunity for networking and socializing over delicious food in the Open Space at 60 5th Avenue.

CDS recently hosted an engaging MaD seminar with Prof. Zhuoran Yang from Yale University. In his talk, "Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality," Prof. Yang delved into the dynamics of gradient flow for training multi-head softmax attention models.

He established the global convergence of gradient flow under suitable initialization choices and proved the emergence of a "task allocation" phenomenon, where each attention head focuses on solving a single task within the multi-task model. Prof. Yang described the three phases of gradient flow dynamics: a warm-up phase, an emergence phase, and a convergence phase.

Prof. Yang also emphasized the significance of this joint work with Yale colleagues Siyu Chen, Heejune Sheen, and Tianhao Wang.

Aram-Alexandre Pooladian, a PhD student at CDS, along with co-authors Yiheng Jiang from NYU’s Courant Institute of Mathematical Sciences, and Sinho Chewi from the Institute for Advanced Study, have proposed a new technique to make big data analysis more efficient. In the paper “Algorithms for Mean-Field Variational Inference via Polyhedral Optimization in the Wasserstein Space,” the authors developed a specific notion of polyhedra to create a simpler, finite-dimensional version of an otherwise complex, infinite-dimensional space. “The core idea of our work is to create a simpler subset over which to optimize. This makes the optimization procedure tractable and still gives good results with precise guarantees” Pooladian explains.

Reflecting on his experience at CDS, Pooladian emphasizes the collaborative and supportive environment. “The collaborative spirit and access to diverse expertise at CDS and Courant have been invaluable. It’s an ecosystem that fosters innovation and rigorous research.”

Discover more about this innovative research and its impact:

New Techniques from CDS Make Big Data Analysis More Efficient CDS PhD student Aram-Alexandre Pooladian’s new technique enhances big data analysis.

CDS Lunch Seminar: Umang Bhatt on Algorithmic Resignation

CDS Assistant Professor and Faculty Fellow Umang Bhatt presented his research on algorithmic resignation at a recent CDS Lunch Seminar. Bhatt, who joined CDS in the fall after completing his PhD at the University of Cambridge, studies how to create AI decision-making systems that can explain their behavior and adapt to stakeholders' expertise.

In his talk, Bhatt discussed the concept of algorithmic resignation — the deliberate disengagement of AI systems in certain settings based on factors like user expertise, cost, and regulations. He argued AI access should be personalized, with systems "resigning" for some users in contexts where human judgment is more appropriate.

Bhatt shared examples of AI resignation policies, such as a law firm restricting usage for client communications but allowing it for internal paralegal work. He also presented an approach to learn personalized AI support policies based on users' performance on tasks with and without AI assistance.

"Resignation is a mechanism for orchestrating humans and machines to improve outcomes," said Bhatt. "It allows for building trustworthy and compliant AI systems that augment rather than replace human decision-making."

CDS students showcased their ingenuity and passion for machine learning at the recent DS-GA 1003 (machine learning course) Poster Session. The event featured an array of impressive projects, with students exploring state-of-the-art methods, fine-tuning pre-trained models, investigating learned features, and developing new theories.

Working in groups and independently, the students dove deep into topics like loan default prediction, model stability analysis, sentiment analysis, and more. Their posters demonstrated a strong grasp of advanced concepts and a drive to push the boundaries of what's possible in the field.

Congratulations to all the participants on their outstanding work! The DS-GA 1003 Poster Session exemplified the spirit of curiosity and innovation that defines CDS.

New research at CDS challenges the efficacy of state-space models as a robust solution for state tracking tasks. Led by CDS PhD student William Merrill, the study, titled “The Illusion of State in State-Space Models” scrutinizes these models' computational power, juxtaposed against transformers and recurrent neural networks.

With contributions from the Allen Institute for AI's Ashish Sabharwal and fellow NYU PhD student Jackson Petty, this work reveals surprising theoretical limitations of newly proposed models. The authors also suggest possible hybrid solutions.

The Illusion of State: Uncovering the Limitations of State-Space Models CDS’s latest study exposes surprising limitations of state-space models for state tracking—and a potential solution

CDS students gathered for an insightful Pizza Chat with Michael King, CEO/Founder of digital marketing agency iPullRank. King, an award-winning marketing thought leader and technologist, shared his expertise on generative AI in Google & Bing models and entrepreneurship.

King discussed how his agency leverages market segmentation, content strategy, SEO, generative AI, and engineering to create powerful marketing campaigns for Fortune 500 clients. He emphasized the importance of understanding customer psychology and translating it into actionable data.

The engaging Q&A session covered topics ranging from the future of search engines to practical advice for aspiring data science entrepreneurs.

Researchers at CDS have unveiled that transformer language models can solve complex problems without clear intermediate reasoning, challenging existing assumptions about their operation.

Led by CDS PhD students Jacob Pfau and William Merrill, and CDS Associate Professor of Linguistics and Data Science Sam Bowman, this study highlights the use of nonsensical filler tokens that achieve perfect accuracy in specific tasks, raising concerns about interpretability and accountability in LLMs.

Explore the full findings:

Language Models Can Perform Complex Computations Without Interpretable Intermediate Reasoning… CDS research finds transformers can solve complex tasks without interpretable reasoning steps, raising concerns about accountability in…

Congrats to the Class of 2024! 🎓

CDS celebrated the remarkable achievements of our 2024 graduates at joyous ceremonies over the past week. Undergraduates, MS students, and PhD students came together, ready to shape the future of data science.

The events featured speakers like Carlos Fernandez-Granda, Associate Professor of Mathematics and Data Science and CDS’ Interim Director; Pascal Wallisch, Clinical Associate Professor of Data Science and Psychology; Cristina Savin, Assistant Professor of Neural Science and Data Science; and Louis Mittel, Clinical Assistant Professor & Director of Undergraduate Studies, who shared their wisdom.

As the graduates embark on their next chapters, they carry with them the skills and knowledge gained at CDS. We couldn’t be more proud.

Recent CDS MSDS graduate Xiangyue (Max) Wang and his soon-to-be PhD advisor Thomas Morstyn, Associate Professor at Oxford University, have published a paper in Joule exploring how quantum computing could help solve optimization problems for net-zero power systems. The paper reviews the latest work on quantum computing for combinatorial power system optimization problems and maps theoretical work to underexplored applications.

Wang's journey into climate solutions began during the COVID-19 pandemic. He decided to dedicate his career to climate solutions and designed a self-directed curriculum at CDS that specifically focused on courses with applications in climate solutions.

As Wang gears up for his PhD at Oxford, he remains committed to pushing the boundaries of quantum computing for power system optimization, focusing on the intersection of quantum physics and machine learning.

Quantum Computing Paves the Way for a Clean Energy Future Recent CDS MSDS graduate Xiangyue (Max) Wang’s research explores how quantum computing could revolutionize the optimization of power…

CDS recently hosted a Faculty/Student Lunch with CDS Assistant Professor of Psychology and Data Science Grace Lindsay. Prof. Lindsay discussed her research using artificial neural networks to understand the brain, with a particular focus on studying the control and effects of attention on sensory processing.

She also touched on her interest in applying data science to real-world problems, especially in the domain of climate change. "It's a lot of little things," Prof. Lindsay said, when asked about the biggest levers data science can pull in the climate space. "Any area where you could make a process more efficient, or have people make better decisions, because they have a better understanding of the situation—that helps."

Prof. Lindsay is the author of "Models of the Mind: How physics, engineering and mathematics have shaped our understanding of the brain." MS and PhD students engaged in a lively Q&A session on topics ranging from her research to science communication and work-life balance in academia.

CDS hosted an exclusive Fireside Chat with CBRE, providing students with a unique opportunity to engage with distinguished CBRE representatives. The intimate gathering offered firsthand insights into CBRE's mission, values, and innovative data-driven approaches in the real estate industry. Attendees had the opportunity to learn about the company's culture, projects, and the dynamic landscape of real estate services. CBRE is the world's largest commercial real estate services and investment firm.

Following the chat, students participated in a speed networking session, connecting with like-minded individuals passionate about data science. The event fostered meaningful connections and facilitated discussions about classes, projects, and opportunities within the field.