Is ChatGPT (or any other AI) a good tool for summarizing articles?
I’m taking a psychology class and it’s just a list of articles with no lecture, no slides and no teacher notes or even highlights. Just a pile of articles with dozens of pages each.
I really want to use ChatGPT to help summarize these articles and make the reading load potentially easier but I’m worried about the AI hallucinating and potentially leaving me worse off for studying.
depends on how much u care about learning the course material tbh. there’s less chance of hallucinations if you ask for a high level summary but at that point you may as well have just skim read everything.
one thing i would consider is prompting several different models, the main alternatives you can try are gemini-pro from Google and claude-2.1. if they all tell you different things then u probably want to pay attention and just read.
anecdotally: gemini kind of sucks but has multi-modal stuff. claude-2.1 seems better than gpt3.5 across the board.
you could try looking into the CoT prompting all the LLM nerds do, but that stuff tends to work better for asking language models to do a task, and not summaries.
actually, an extra thing to add is that language models tend to ‘pay less attention’ to the middle of the context you send them, and this behaviour gets worse the more tokens they process. if you send them a 4000 word reading ur assigned the language model is not going to recall details in the middle 2000 words as accurately, and possibly just ignore them. this means that for accuracy you may be better off summarising each page one by one, which seems like a pretty simple ctrl+a ctrl+c crtl+v operation but regardless.
if this all sounds very stupid its because they are.
(i didn’t go through the education system with language models but i abuse them heavily for corporate stuff at work. i also am not a cs/ml expert.)
Is ChatGPT (or any other AI) a good tool for summarizing articles?
I’m taking a psychology class and it’s just a list of articles with no lecture, no slides and no teacher notes or even highlights. Just a pile of articles with dozens of pages each.
I really want to use ChatGPT to help summarize these articles and make the reading load potentially easier but I’m worried about the AI hallucinating and potentially leaving me worse off for studying.
Usually im against the use of LLMs, but that class sounds horrible. An LLM would be helpful as a primer before you read.
depends on how much u care about learning the course material tbh. there’s less chance of hallucinations if you ask for a high level summary but at that point you may as well have just skim read everything.
one thing i would consider is prompting several different models, the main alternatives you can try are gemini-pro from Google and claude-2.1. if they all tell you different things then u probably want to pay attention and just read. anecdotally: gemini kind of sucks but has multi-modal stuff. claude-2.1 seems better than gpt3.5 across the board.
you could try looking into the CoT prompting all the LLM nerds do, but that stuff tends to work better for asking language models to do a task, and not summaries.
actually, an extra thing to add is that language models tend to ‘pay less attention’ to the middle of the context you send them, and this behaviour gets worse the more tokens they process. if you send them a 4000 word reading ur assigned the language model is not going to recall details in the middle 2000 words as accurately, and possibly just ignore them. this means that for accuracy you may be better off summarising each page one by one, which seems like a pretty simple ctrl+a ctrl+c crtl+v operation but regardless. if this all sounds very stupid its because they are.
(i didn’t go through the education system with language models but i abuse them heavily for corporate stuff at work. i also am not a cs/ml expert.)