Consensus - ChatGPT for Science

Potentially interesting tool given the natural interest in physiology and endurance training in this forum

Ask a scientific question and Consensus AI will deliver summary of top papers. papers. Naturally, powered by GPT-4.

Registration required.

1 Like

Did you intentionally write papers twice? Sorryā€¦couldnā€™t resist.

goodfellas-two

3 Likes

I donā€™t have an account, out of curiosity, what does it say if the question is ā€˜Is pyramidal training better for recreational athletes?ā€™

1 Like

No useful result.

*Not enough relevant results. Try asking a well-researched question in the format ā€˜Does x cause y?ā€™"

PAPERS!

(no - not intentionally - lol)

1 Like

Unclear how relevant this is, but an attorney in my office has been playing around with ChatGPT for motion writing and they reported that it does a great job with one major caveat: it seems to also generate itā€™s own made up case law, making it useless.

If it does the same thing with summaries of science research, it would be similarly useless.

This could be total user error on my colleagueā€™s part, but I thought it was interesting. Sooner or later itā€™ll definitely replace a good portion of my work, which will be really nice.

5 Likes

No doubt that better prompting also leads to better answers. That being said, ā€˜AI hallucinationsā€™ is very much a feature of current generation Large Language Models.

By 2023, analysts considered frequent hallucination to be a major problem in LLM technology, with a Google executive identifying hallucination reduction as a ā€œfundamentalā€ task for ChatGPT competitor Google Bard.[6][23] A 2023 demo for Microsoftā€™s GPT-based Bing AI appeared to contain several hallucinations that went uncaught by the presenter.[6]

Hallucinations are not well understood. This is going to be quite a rideā€¦

The hallucination phenomenon is still not completely understood.[2] Therefore, there is still ongoing research to try to mitigate its apparition.[25] Particularly, it was shown that language models not only hallucinate but also amplify hallucinations, even for those which were designed to alleviate this issue.[26]

Iā€™ve been playing with Google Bard a little and itā€™s interesting to ask it things that may or may not be controversial that I already have an opinion about. Examples - is a low carb diet good for cyclists, or what is the ideal weight for a cyclist that is 6ā€™4.

Itā€™s kind of fun but itā€™s answering in ways that seem like common knowledge despite what some sections of the internet would like you to believe, but if I was asking about something that I wasnā€™t familiar with Iā€™m not sure how much I would trust the results.

I guess itā€™s an interesting thing to play with but I think when people start using this stuff for work or to replace people we are going to get news about glaring and obvious errors.

1 Like

Hahaha :smiley:

2 Likes

Easier to try GPT SCI

No registration is required.

Chat GPT did my ramp test but now I have to deal with a 678 FTP.

9 Likes

Very early days still for these Large Language Models and the tools that are being developed to leverage them. They will become more useful over time and the speed of the development might be very impressive.

That said, the roadmap for chatGPT doesnā€™t seem to involve actually pushing the pedals yet, so we humans are still going to have something to do in order to make our bikes move more quickly for longerā€¦

Iā€™ve tried ChatGPT for various technical things like writing code for network routers and it writes a load of plausible looking stuff that would fool anyone that didnā€™t know that it was basically talking a load of crap.

4 Likes

I do a ton of technical report writing for my job (environmental consultant). One of my colleagues has been obsessed with chat GPT, saying itā€™s saving him tons of time, etc. Yesterday he asked me to review a discussion section for a report. It was absolute garbage, completely unacceptable to send to a client. When I asked him he came clean and said he just used chat GPT. Same experience as you, it provides lots of plausible and informed sounding sentences, but if you have a deep knowledge of the subject matter itā€™s trash.

1 Like

It does a poor job drawing conclusions and summarizing but if you have it elaborate on each point, one by one in separate prompts, and provide it with some of your expertise, it often figures it out and aligns. It can be extremely powerful in this way.

It needs better prompting and better followup prompting.

For example, if you tell it ā€œwrite me some database code to do blah blah blahā€, itā€™ll spit out something within the token limit that may or may not be shit.

But if instead you go, ā€œwrite me a MySQL query using PDO prepared statements in PHP. Hereā€™s my table schema ______. Use PHP filter_var, etc.ā€ Itā€™ll actually create some very usable code that can save you hours every day.

1 Like

This is the best explainer Iā€™ve seen for LLMs for people who donā€™t have a technical (read: machine learning) background: Chat GPT is a Blurry JPEG of the Web

And yes, ā€œAI hallucinationā€ is a thing. It would be better if we called it what it was - which is that LLMs make up information based on statistical relationships between words, but this is where we are right nowā€¦

Totally agree.

I am using ChatGPT within my field of work. ChatGPT can certainly produce a lot of plausible sounding garbage without the right prompting.

With better and more specific prompting (including use of system personas) the quality improves drastically. But asking the right questions within a field where you donā€™t have much knowledge can be tricky.

I downloaded data from intervals.icu - I had a list of the best 20 minute, 30 minute and 60 minute power from all my rides over the last 2 years.

I uploaded this to chatgpt and asked for some insights and for a graph to show the best outputs from each for each 3 month period (to simply the data) and to plot the estimated ftp from the data. I thought it was really interesting and useful. For me, understanding how I have actually improved is huge to keep me going. This clearly shows a lovely improvement and was very easy to get thanks to Chatgpt.

1 Like

Off topic, how did you get this output?

Nice result!