DMR’s AI Models more accurate than GPT-4 in Sentiment Analysis
Until we get to artificial general intelligence (AGI) - if ever - training data created by humans will be king if we want machine learning models that perform with at least 80% accuracy.”
LONDON, UNITED KINGDOM, October 31, 2023 /EINPresswire.com/ -- The DMR data science team performed an accuracy gap analysis between GPT-3.5 Turbo and DMR's custom AI models for sentiment tagging in August 2023. The results were striking, with DMR outperforming GPT-3.5 Turbo by a substantial 27 percentage points in the zero-shot scenario.— Michalis Michael - DMR CEO
DMR obtained access to GPT-4 this month after a lengthy process, primarily due to the rather cumbersome process of OpenAI and has performed a similar gap analysis as with GPT3.5 turbo.
DMR’s gap analysis on GPT-4 showed that the average F1 Score gap between DMR's AI sentiment model and GPT-4 is 11 percentage points in favour of DMR. While this is a 16-percentage point improvement over GPT-3.5 Turbo, it still falls short of DMR's performance by a double-digit percentage.
This is another proof to support multiple peer reviewed scientific papers which state that LLMs have a disadvantage compared to context based custom AI models trained on data that relate to the subject of the research. In this case, the data used were banking and alcoholic beverages related posts gathered from social media and other online posts using listening247 – the unstructured data analytics platform of DMR.
Michalis Michael
DMR
+44 20 3795 4715
email us here
Visit us on social media:
Facebook
Twitter
LinkedIn
YouTube
Legal Disclaimer:
EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.
