TPC Logo reads "Trillion Parameter Consortium. AI For Science"

TPC European Kick-Off Workshop

European Kick-Off of the International Trillion Parameter Consortium (TPC): Accelerating the development and use of Generative AI for Science and Engineering

Workshop Home

Program

Breakouts

Tutorial

Speakers

Background

The predictive power of Large Language Models (LLMs) has increasingly made them go-to methods across many scientific domains. This presents scientists with new challenges but even greater opportunities as they explore various approaches and drive changes in data protection, data usage, and scientific understanding. The international Trillion Parameter Consortium (TPC) aims to bring together groups interested in collaborating around important areas including building, training, and using large-scale AI models as well as building and operating large-scale computing systems. TPC convenes individuals from three broad and overlapping communities: (1) those working on AI methods development, natural language processing/multimodal approaches and architectures, full stack implementations, scalable libraries and frameworks, AI workflows, data aggregation, cleaning and organization, training runtimes, model evaluation, downstream adaptation, alignment, etc.; (2) those who design and build hardware and software systems; and (3) those who will ultimately use the resulting AI systems to explore a range of challenges in science, engineering, medicine, and other domains.

This workshop will provide a forum for continuing to develop a shared vision and goals, particularly aimed at accelerating the use of generative AI in science and engineering. It offers an opportunity for participation by the European AI, HPC, and disciplinary science research communities. On the first day of the meeting, attendees can participate in a hands-on tutorial focused on Large Language Models for Science and Engineering Applications.

Workshop Agenda

The workshop agenda comprises thematic plenary sessions and parallel workshop sessions organized around multiple themes, each featuring lightning talks, panels, and group discussions. These sessions aim to facilitate the formation and deepening of collaborations, explore opportunities to coordinate efforts, and strengthen collaborations within the European AI, HPC, and Science communities. The objective of the TPC workshop program is to advance the building, evaluating, and use of large-scale AI models for science and engineering applications. Each theme is organized by individuals active in corresponding TPC working groups, with participants from around the globe who have been collaborating virtually since 2023. Parallel breakout sessions will include lightning talks on cooperative efforts, lessons learned to date, and discussions.

The LLM tutorial is optional, with those not attending the tutorial arriving for lunch on 19-June at 14:00, followed by an opening plenary at 15:00 and a welcome reception in the evening. The workshop will run until 13:45 on Friday 21-June.

Background: Why TPC?

Today the science community has a growing number of projects aiming to harness generative AI, ranging from models trained for individual disciplines (e.g., UniverseTBD for astronomy or MAIRA-1 for radiological image analysis) to models aspiring to multi-disciplinary use (e.g., Olmo, AuroraGPT). There are also new data sources being released, such as DOLMA, and many efforts to identify scientific data (literature, databases, time series data, etc.) and transform those data for use training AI models.  There has been an explosion of new activities, each developing methods and tools to grapple with common challenges, from evaluation (for trustworthiness, safety, bias, performance, etc.) to training and data preparation workflows.  Concurrently, the enormous cost of computation for model training limits the number of groups that can realistically build and train large-scale models.  These trends all offer incredible opportunities to collaborate, both to accelerate progress toward optimal tools and methods and to enable groups to strategically collaborate to reduce duplication of effort, such as in data preparation of new scientific data sources. Additionally, many new challenges are coming to the fore with generative AI, including new ways of thinking about data sharing and attribution, about licensing artifacts, and about the importance of responsible development of safe, ethical AI models.  An overarching need today in AI is openness – which includes open data, open source code for tools and workflows, and careful thought as to how, when, and whether to open the models themselves. Such openness is critical to progress in every area related to generative AI.

The enormity of these challenges, and of the resources needed for data preparation, pre-training new models, and responsibly preparing them for downstream applications has meant that progress is largely concentrated in industry, where there is limited, or in some cases, no visibility into the artifacts (models, data sets) or the processes used to create them.  This underscores the need for collaboration in the open science community—central to the motivation behind creating the international Trillion Parameter Consortium (TPC). The workshop aspires to stimulate new thinking, attract scientists to some of the emerging challenges associated with generative AI, and potentially catalyze the formation of new topical collaborative working groups that can be supported by the TPC.

TPC: Expected Impact

The trends discussed above and the science community’s quest to use generative AI for science require collaborations among HPC and AI experts, domain scientists, and those at HPC centers providing data and computational resources to discuss current trends and breakthroughs and explore potential collaborations.  The international Trillion Parameter Consortium (TPC) was formed in 2023 for this very purpose—to bring this diverse community together to identify and pursue collaborations to accelerate progress and to enable the broadest possible community to benefit from the limited HPC, data, and human resources available to train large AI models and to develop the necessary tools and methods for fully leveraging AI for scientific and engineering applications.

TPC also aims to grow the community itself, both assisting scientists in learning how to exploit AI for their research and enabling early- and mid-career scientists to take on leadership roles in the community. TPC participation ranges from HPC providers to computer and computational scientists to disciplinary scientists and encompassing those involved in career development and training programs.   

Program Committee

  • Jean-Yves Berthou, Inria
  • Jérôme Bobin, CEA
  • Fabrizio Gagliardi, Barcelona Supercomputing Center
  • Kimmo Koski, CSC – IT Center for Science
  • Dieter Kransmüller, Leibniz Supercomputing Centre
  • Laura Morselli, CINECA
  • Per Öster, CSC – IT Center for Science
  • Gabriella Scipione, CINECA
  • Alfonso Valencia, Barcelona Supercomputing Center
  • Miguel Vazquez, Barcelona Supercomputing Center
  • Prasanna Balaprakash, Oak Ridge National Laboratory
  • Charlie Catlett, Argonne National Laboratory and University of Chicago
  • Franck Cappello, Argonne National Laboratory
  • Murali Emani, Argonne National Laboratory
  • Ian Foster, Argonne National Laboratory and the University of Chicago
  • Jason Haga, AIST
  • Eliu Huerta, Argonne National Laboratory
  • Volodymyr Kindratenko, University of Illinois
  • Neeraj Kumar, Pacific Northwest National Laboratory
  • Satoshi Matsuoka, RIKEN
  • Rajeev Thakur, Argonne National Laboratory
  • Michael Papka, Argonne National Laboratory and University of Illinois Chicago
  • Noah Smith, Allen Institute for Artificial Intelligence and the University of Washington
  • Rick Stevens, Argonne National Laboratory and the University of Chicago
  • Valerie Taylor, Argonne National Laboratory and the University of Chicago
  • Mohamed Wahib, RIKEN
  • Rio Yokota, Tokyo Institute of Technology