As noted in the July 2023 TPC announcement, TPC has grown out of a number of interactions during the past several years. As it became clear that a collaborative initiative like TPC was critically needed, leaders from multiple organizations convened a workshop in August 2023.

Some 200 people from over 50 organizations around the world gathered for a hybrid meeting hosted by Argonne National Laboratory on 2-3 August 2023. The purpose was to explore areas for collaboration toward building large-scale generative AI models. The meeting hosted three tracks of discussions: data, models, and crosscuts. The data track comprised five discipline-specific working groups, each focused on bringing together and preparing data from those disciplines (loosely defined, for instance biology-biochemistry-drugs and materials-chemistry-nanoscience). The models track included working groups effectively represented stages in the pipeline from model architecture to pre-training, optimization, evaluation, and downstream applications. Finally, two groups made up the crosscuts track. One began to flesh out organizational, logistical, governance, and policy concepts for TPC, including issues such as openness, licensing options in context of open data and open models. The second group discussed opportunities for fundamental computer science contributions to the data and modeling efforts, including the use of computer science expertise and techniques to address critical generative AI challenges ranging from training performance optimization to generative AI model “hallucinations.”

Information and outcomes from this first TPC meet-up, including introductory slides about TPC and slides from each working group, have been posted in the TPC Slack workspace. (join us there!)

On-site participants. We also thank those not shown here: More than 130 remote participants, including session leaders from other continents working at quite inconvenient hours in their local timezones!

Trending

Discover more from Trillion Parameter Consortium (TPC)

Subscribe now to keep reading and get access to the full archive.

Continue reading