ChatGPT gets code questions wrong 52% of the time
ChatGPT gets code questions wrong 52% of the time
But its suggestions are so annoyingly plausible
You're viewing a single thread.
Title feels misleading, it gets stack overflow questions wrong 52% of the time
However it got 77% of easy Leetcode questions correct. Also I believe that's first try, which is not generally how chatgpt should be used.
Also also, you should probably be using a coding specific model if you want good coding results
3 1 ReplyEvery leetcode question has been answered a billion times and you train it on those billions of answers, it should get those right.
5 0 ReplyProbably because the model has seen thousands of possible solutions to those exact Leetcode problems. Actual questions people ask on StackOverflow tend to be much more specialized.
3 0 ReplyBut it confidently explains the wrong answers.
I just hope politicians don't find out how to use it. It'll be our doom.
2 0 Reply