dc.contributor.author | Ganesan, Adithya V | |
dc.contributor.author | Lal, Yash Kumar | |
dc.contributor.author | Nilsson, August | |
dc.contributor.author | Schwartz, H. Andrew | |
dc.date.accessioned | 2024-02-29T12:50:58Z | |
dc.date.available | 2024-02-29T12:50:58Z | |
dc.date.created | 2024-01-17T18:04:45Z | |
dc.date.issued | 2023 | |
dc.identifier.isbn | 978-1-959429-87-6 | |
dc.identifier.uri | https://hdl.handle.net/11250/3120496 | |
dc.description.abstract | Very large language models (LLMs) perform extremely well on a spectrum of NLP tasks in a zero-shot setting. However, little is known about their performance on human-level NLP problems which rely on understanding psychological concepts, such as assessing personality traits. In this work, we investigate the zero-shot ability of GPT-3 to estimate the Big 5 personality traits from users’ social media posts. Through a set of systematic experiments, we find that zero-shot GPT-3 performance is somewhat close to an existing pre-trained SotA for broad classification upon injecting knowledge about the trait in the prompts. However, when prompted to provide fine-grained classification, its performance drops to close to a simple most frequent class (MFC) baseline. We further analyze where GPT-3 performs better, as well as worse, than a pretrained lexical model, illustrating systematic errors that suggest ways to improve LLMs on human-level NLP tasks. The code for this project is available on Github1
. | en_US |
dc.language.iso | eng | en_US |
dc.publisher | Association for Computational Linguistics | en_US |
dc.relation.ispartof | The 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis | |
dc.relation.ispartofseries | ACL Anthology; | |
dc.rights | Navngivelse 4.0 Internasjonal | * |
dc.rights.uri | http://creativecommons.org/licenses/by/4.0/deed.no | * |
dc.title | Systematic Evaluation of GPT-3 for Zero-Shot Personality Estimation | en_US |
dc.type | Chapter | en_US |
dc.type | Peer reviewed | en_US |
dc.type | Conference object | en_US |
dc.description.version | publishedVersion | en_US |
cristin.ispublished | true | |
cristin.fulltext | original | |
cristin.fulltext | original | |
cristin.qualitycode | 1 | |
dc.identifier.doi | https://aclanthology.org/volumes/2023.wassa-1/ | |
dc.identifier.cristin | 2228949 | |