Written in Dec 2023 - Jan 2024
Zhen Wang, co-writing with Fan Bai
Comments from Zhiting Hu@UCSD and Yoon Kim@MIT
Gentle disclaimers: This blog partially focuses on the NLP side, specifically large language models (LLMs). Note that this is not a comprehensive review of the development of NLP or LLMs. We focus on research directions that are heavily impacted by the release of ChatGPT and highlight the twists.
Released on November 30, 2022, during the NeurIPS'22 conference, ChatGPT rapidly became a focal point of awe and discussion due to its remarkable text-generation capabilities and versatility in handling diverse text-based tasks. The year 2023 witnessed ChatGPT undergoing significant changes, including the enhancements of GPT-4, Plugins, Browsing, Code Interpreter, Dalle-3, and other advancements.
The evolution of ChatGPT has sparked numerous new research directions in NLP and beyond, igniting dramatic, thought-provoking discussions. The AI field's potential for high profitability has often led to lofty expectations around certain aspects of ChatGPT research, for example, the long-desired AGI. However, the landscape has also seen unexpected shifts in interest and focus, sometimes occurring within mere months.
With the conclusion of another NeurIPS conference (Dec 10-16, 2023), it's an opportune moment to reflect on the achievements and missed targets of ChatGPT and the broader community in its inaugural year. Such retrospection is vital for understanding the fluctuations in expectations (denoted as “twists” in the following context) and perceptions of ChatGPT's research trajectory.
In this context, we define a "twist" as a research direction that has seen a surprising surge or decline in interest from the academic community post-ChatGPT's release. This contrasts with the industry's possibly differing reactions. Simultaneously, there are also "smooth" research directions that have maintained their significance both before and after ChatGPT's debut. Note that a “twist” or research problem shift might be temporal and also very common in research, reflecting the fast-changing and nuanced nature of technological advancement.
The following topics are not ranked by their priorities. Each topic/direction is presented in the following format:
Brief Introduction
Expectation (Expectation at the beginning of 2023)
Twist (Realization after ~12 months)