• Feedback
  • RSS Feed
  • Sitemap
Ommcom News
  • Home
  • Odisha
  • Nation
  • World
  • Sports
  • Business
  • Entertainment
  • Videos
  • Science & Tech
  • Photo Gallery
  • ଓଡ଼ିଆରେ ପଢନ୍ତୁ
No Result
View All Result
  • Home
  • Odisha
  • Nation
  • World
  • Sports
  • Business
  • Entertainment
  • Videos
  • Science & Tech
  • Photo Gallery
  • ଓଡ଼ିଆରେ ପଢନ୍ତୁ
No Result
View All Result
Odisha News, Odisha Breaking News, Odisha Latest News || Ommcom News
Home Science & Tech

ChatGPT’s Answers To Software Engineering Questions Were 52% Incorrect

OMMCOM NEWS by OMMCOM NEWS
August 13, 2023
in Science & Tech
ChatGPT

New York:  OpenAI’s ChatGPT answered about 52 per cent software engineering questions incorrectly, according to a study, raising questions about the popular language models accuracy.

Despite ChatGPT’s popularity, there hasn’t been a thorough investigation into the quality and usability of its responses to software engineering queries, said researchers from the Purdue University in the US.

To address this gap, the team undertook a comprehensive analysis of ChatGPT’s replies to 517 questions from Stack Overflow (SO).

“Our examination revealed that 52 per cent of ChatGPT’s answers contain inaccuracies and 77 per cent are verbose,” the researchers wrote in the paper, not peer-reviewed and published on a pre-print site.

Importantly, the team found that 54 per cent of the time the errors were made due to ChatGPT not understanding the concept of the questions.

Even when it could understand the question, it failed to show an understanding of how to solve the problem, contributing to a high number of conceptual errors, they said.

Further, the researchers observed ChatGPT’s limitation to reasoning.

“In many cases, we saw ChatGPT give a solution, code, or formula without foresight or thinking about the outcome,” they said.

“Prompt engineering and human-in-the-loop fine-tuning can be helpful in probing ChatGPT to understand a problem to some extent, but they are still insufficient when it comes to injecting reasoning into LLM. Hence it is essential to understand the factors of conceptual errors as well as fix the errors originating from the limitation of reasoning,” they added.

Moreover, ChatGPT also suffers from other quality issues such as verbosity, inconsistency, etc. Results of the in-depth manual analysis pointed to a large number of conceptual and logical errors in ChatGPT answers. The linguistic analysis results showed that ChatGPT answers are very formal, and rarely portray negative sentiments.

Nevertheless, users still preferred ChatGPT’s responses 39.34 per cent of the time due to its comprehensiveness and articulate language style.

“These findings underscore the need for meticulous error correction in ChatGPT while also raising awareness among users about the potential risks associated with seemingly accurate answers,” the researchers said.

(IANS)

Tags: ChatGPT
ShareTweetSendSharePinShareSend
Previous Post

Storms Can Last For 100 Years On Saturn: Study

Next Post

Nick Jonas Looks Back At An Awkward Moment In His Marriage Ceremony

Related Posts

Atal Pension Yojna
Science & Tech

Atal Pension Yojna Surpasses 8 Crore Enrollments, Adds 39 Lakh New Members In FY26

July 25, 2025
Science & Tech

India’s Edge Data Centre Capacity Projected To Triple To 200-210 MW By 2027

July 24, 2025
Science & Tech

7,000 Daily Steps Can Cut Your Risk Of Cancer, Depression, Death: The Lancet

July 24, 2025
Rare Earth Elements
Science & Tech

India Has 8.52 Million Tonnes Reserves Of Rare Earth Elements: Jitendra Singh

July 23, 2025
Science & Tech

Lung TB: Higher Dose Of Rifampicin Safe, Can Boost Recurrence-Free Survival, Says ICMR Study

July 17, 2025
Science & Tech

Axiom Space Celebrates The Arrival Of Ax4 Mission

July 16, 2025
Next Post
Priyanka Chopra

Nick Jonas Looks Back At An Awkward Moment In His Marriage Ceremony

WhatsApp Rolling Out Community Examples On iOS Beta

Parliamentary Panel Seeks Sunset Clause For Cesses, Periodic Review

Parliamentary Panel Seeks Sunset Clause For Cesses, Periodic Review

khimji
TPCODL
OMC
  • Feedback
  • RSS Feed
  • Sitemap

© 2025 - Ommcom News. All Rights Reserved.

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In

Add New Playlist

No Result
View All Result
  • Home
  • Odisha
  • Nation
  • World
  • Sports
  • Business
  • Entertainment
  • Videos
  • Science & Tech
  • Photo Gallery
  • ଓଡ଼ିଆରେ ପଢନ୍ତୁ

© 2025 - Ommcom News. All Rights Reserved.