Welcome to the PyData Pittsburgh Forum! If you're new to the community, please start here.
A Tour of Large Language Models: An Accessible Journey into How They Work
-
Please join PyData Pittsburgh for a tour of large language models (LLMs) with Jay Palat. We'll be gathering at COhatch Waterfront, a brand-new coworking space at the Waterfront in Homestead.
Please RSVP on the Meetup.com event listing here:
https://www.meetup.com/pydata-pittsburgh/events/293765117/
About the talk
Every day, the landscape of large language models (LLMs) expands with the unveiling of new models from OpenAI, Google, Microsoft, Anthropic, Meta and others. These companies are making substantial investments in harnessing the capabilities of LLMs, as evidenced by the array of cutting-edge features showcased at their developer conferences. The applications of LLMs span from assisting in code writing to generating awe-inspiring narratives and enabling versatile chatbots.
Yet, amidst this wave of excitement, one question lingers: How do these Large Language Models actually work? In this talk, we aim to provide a comprehensible overview of the inner workings of LLMs, tracing their origins and delving into the challenges associated with their training and deployment. By attending this talk, you will gain an understanding of these transformative tools, empowering you to strategically employ them in solving the problems you encounter. Join us on an accessible journey into the functionality and potential of Large Language Models.
(*In the spirit of the talk, this abstract was enhanced with ChatGPT.)
About Jay
Jay Palat is a seasoned technical leader with expertise in human-centered emerging technologies. His recent work includes working to build the discipline of AI Engineering for safety and mission critical AI systems as the Technical Director of AI for Mission at the CMU's Software Engineering Institute AI Division. Jay has built a career helping teams engineer good solutions that solve complex problems with companies like IBM, UPMC Enterprises, Rhiza and BCG. When he's not working or with his family, Jay's often walking the parks and streets of Pittsburgh.
-
Thanks again to everyone who braved the pouring rain and intermittent Meetup.com outages to come to the event! I thought we had a great turnout and a lively discussion.
Jay has made the slides from his presentation available here:
https://docs.google.com/presentation/d/1-3q-zgF15IxkXkpg0FlzSNUdRmEvNT79iVS8j7NREbM/edit?usp=sharing
-
One topic that came up in Jay's talk was reinforcement learning with human feedback (RLHF). For anyone who would like to go deeper on the topic, here are two sources I'd recommend:
Andrej Karpathy's "State of GPT" talk
https://www.youtube.com/watch?v=bZQun8Y4L2A
Chip Huyen's RLHF explainer
