The Trillion Parameter Consortium (TPC) has witnessed incredible interest, along with many questions about our consortium goals and operations. This post lays out the purpose of the consortium and an initial process for leveraging current momentum and interest.

Goals of the Consortium. Broadly speaking, the TPC has three goals:

• Goal 1. Build an open community of researchers that are interested in creating state-of-the-art large-scale generative AI models (e.g., Foundation Models, Large Language Models) aimed broadly at advancing progress on scientific and engineering problems, by sharing methods, approaches, tools, insights, and workflows.
• Goal 2. Incubate, launch, and facilitate coordination and collaboration on projects to build specific models at specific sites, striving to avoid unnecessary duplication of effort and to maximize the impact of the projects in the broader AI and scientific community. Where possible, we will work out what we can do together for maximum leverage versus what needs to be done in smaller groups.
• Goal 3. Create a global network of resources and expertise that can help facilitate teaming and training the next generation of researchers in AI and related fields, particularly those interested in the development and use of large-scale AI in advancing science and engineering.

Target Community. The overarching focus of the consortium is to bring together groups interested in building, training, and using large-scale models with those who are building and operating large-scale computing systems. The target community encompasses (a) those working on AI methods development, natural language processing/multimodal approaches and architectures, full stack implementations, scalable libraries and frameworks, AI workflows, data aggregation, cleaning and organization, training runtimes, model evaluation, downstream adaptation, alignment, etc.; (b) those that design and build hardware and software systems; and (c) those that will ultimately use the resulting AI systems to attack a range of problems in science, engineering, medicine, and other domains.

What we are not trying to do. We are not trying to control which projects groups decide to pursue, and we are not trying to determine who collaborates with whom. We are also not recommending specific platforms or approaches.

What are trying to do. Share experiences, tools, data, and code where appropriate and with full consent from participants; make it easier for researchers with common interests to find each other and collaborate; and advocate for best practices in responsible AI development and evaluation where we can identify such practices and where there is consensus.

Operational Model. We expect the consortium to engage in various types of activities depending on the interests of participants, but likely to include:

• Facilitating meetups and hackathons targeting specific goals that support one or more partner projects (e.g., aggregating, cleaning and curating training data, designing a scalable model architecture for a given target platform, collaborating on large-scale model evaluation suites and studies, benchmarking and comparing models);
• Organizing (virtual and face-to-face) seminars and site visits relating to future research directions and open problems in building and evaluating large-scale AI systems for science and engineering;
• Working together to generate white papers or other materials to help advocate and explain the need for advanced AI systems optimized for scientific and engineering use cases;
• Identifying and promoting opportunities for visiting students, post-docs, and researchers for related activities, summer schools or project-related work aimed at large-scale AI for science and engineering; and
• Collaborating to propose, secure, and manage allocations of machine time for group projects that span one or more sites.

Governance. As we boot up the consortium, we will seek individuals that have an interest in helping to lead, coordinate, and manage activities. We will create a Steering Committee composed of a representative of each participating institution who has an interest in helping to nucleate activities and help move efforts forward. We also expect to create working groups around key topics to move the agenda forward. Depending on how many specific model development efforts are launched, these working groups may be involved in one or multiple model development and/or evaluation efforts. As we work out how the group wants to operate, we will plan to work with everyone to develop a governance structure that works.

Get Connected:

• Join the TPC Slack Workspace
• Contribute via GitHub TCP-AI Org