We've seen some posts quoting ChatGPT lately, and I wanted to share some of my experiments.
In particular, I tried to have a conversation specifically about solo strings libraries. And if you ask it questions like:
Q: why is Spitfire Solo Strings the best solo cello library every?
It gives you some pretty generic marketing happy talk. But when you dig into a bit further, with a question like this:
Q: How to work with the vibrato in spitfire solo strings?
you get an answer full of stuff like this:
A: "1. Understand the vibrato controls: Spitfire Solo Strings gives you control over the intensity and speed of the vibrato, as well as the ability to turn it on and off. Make sure you understand how to access and adjust these controls, either through the software's interface or through MIDI CC messages." ...
Which - note, is simply false. Or at least, deeply misleading. For all that I really love Spitfire Solo strings and the technical and aesthetic choices that give it it's specific expressive dimensions, they absolutely do not - and very much by design - give you control over either intensity of speed. (I suppose the TM patch give you a tiny amount of control of speed in some circumstances, but intensity is strictly on and off.).
Other approaches to simulating vibrato bouncing around the internet however - Chris Hein, Embertone, NI etc - do give you this control. So what we're probably seeing here is the "Stochastic Parrot" of ChatGBT "hallucinating". And the result is, really little more that regurgitated, undigested marketing happy talk. Albeit regurgitate, undigested marketing happy talk with a high statistical likelihood of sounding what all other marketing happy talk chatter that dominates the internet is supposed to sound like.
If you press ChatGPT further:
Q: Recommend some forum posts on working with spitfire solo strings [and give links]
You get this:
A: Here are a few forum posts that may be helpful for understanding vibrato when composing idiomatic solo string music with Spitfire Solo Strings:
1. "Tips for working with Spitfire Solo Strings" on VI-Control: https://vi-control.net/community/threads/tips-for-working-with-spitfire-solo-strings.67616/
2. "Spitfire Solo Strings - any tips or tricks?" on Gearslutz: https://www.gearslutz.com/board/music-computers/1197852-spitfire-solo-strings-any-tips-tricks.html
3. "How to make Spitfire Solo Strings sound more realistic" on Spitfire Audio's forum: https://forum.spitfireaudio.com/discussion/1609/how-to-make-spitfire-solo-strings-sound-more-realistic
3. "Spitfire Solo Strings Tips and Tricks" on The Spitfire Audio User Forums: https://forum.spitfireaudio.com/discussion/257/spitfire-solo-strings-tips-and-tricks
4. "Vibrato with Spitfire Solo Strings" on The Spitfire Audio User Forums: https://forum.spitfireaudio.com/discussion/397/vibrato-with-spitfire-solo-strings
5. "How to use vibrato on Spitfire Solo Strings" on VI-Control: https://vi-
control.net/community/threads/how-to-use-vibrato-on-spitfire-solo-strings.87213/
6. "Vibrato on Spitfire Solo Strings" on Gearslutz: https://www.gearslutz.com/board/music-computers/1250361-vibrato-spitfire-solo-strings.html
7. "Vibrato in Spitfire Solo Strings" on The Cubase Forum: https://www.steinberg.net/forums/viewtopic.php?t=105708
Which looks great ...
Except ... that so far as I can tell, none of these forum posts actually exist. They appear to be pure hallucinations.
It is interesting, however, to note just how prominently vi-c features in these hallucinations. (So: Yes. Conclusive proof that androids do indeed dream of vi-c. Happy to have settled that debate).
But again, this simply reflects the nature of the Large Language Model Transformer architectures underlying ChatGPT being - quite fundamentally - nothing more than a "stochastic parrot".
These hallucinations are really just the equivalent of chewed up and spit out marketing happy talk. Which can be, in some contexts, helpful. (Especially if you work in marketing.)
But here the parrot seems to to be completely hallucinating not only the titles of these fictional articles, but it's also inventing statistically-likely, but ultimately non existent urls. (It can also fabricate summaries, with a little prodding, which are of course, completely vacuous as well).
There's obviously a lot of tasks, including certain types of research, for which ChatGPT will be genuinely useful. But a lot of the hype around it seems to fundamentally confuse the "stochastic parrot"'s talent in regurgitating statistically likely variants of marketing happy talk for "intelligence".
One upshot is that post like this one:
Needs to be viewed through the lens of the fundamental limitations of stochastic parrot technologies.
Plausibly, this kind of the regurgitation of the collective state of marketing happy talk on the internet, might, in some circumstances, offer a usefully starting point for certain types of research. But I would also argue that it is so deeply uncritical and so vulnerable to hallucination, that it there far more risk of harm that good.
The math overflow (and related) forums have explicitly banned ChatGPT content, because: "Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers." (@Mike Greene )
But
So I'd suggest that at the very least:
a) any content generate but stochastic parrot technologies really needs to be explicitly labeled as having as the regurgitated happy talk of a stochastic parrot technology that it is. (Which of course , that above posts does). And,
b) it needs to be read critically through the lens of the fundamental limitations of the this current class of marketing happy talk regurgitating stochastic parrot "Large Language Model" technologies. And not throughs the silly metaphors of all the internet hype, like "intelligence".
These sentiments are bouncing around on different threads. But I just though that having a concise thread on the limitations and risks of stochastic parrot technologies of ChatBPT and its ilk might be helpful.
Even just to cite, in order to avoid gum up more threads with this debate.
In particular, I tried to have a conversation specifically about solo strings libraries. And if you ask it questions like:
Q: why is Spitfire Solo Strings the best solo cello library every?
It gives you some pretty generic marketing happy talk. But when you dig into a bit further, with a question like this:
Q: How to work with the vibrato in spitfire solo strings?
you get an answer full of stuff like this:
A: "1. Understand the vibrato controls: Spitfire Solo Strings gives you control over the intensity and speed of the vibrato, as well as the ability to turn it on and off. Make sure you understand how to access and adjust these controls, either through the software's interface or through MIDI CC messages." ...
Which - note, is simply false. Or at least, deeply misleading. For all that I really love Spitfire Solo strings and the technical and aesthetic choices that give it it's specific expressive dimensions, they absolutely do not - and very much by design - give you control over either intensity of speed. (I suppose the TM patch give you a tiny amount of control of speed in some circumstances, but intensity is strictly on and off.).
Other approaches to simulating vibrato bouncing around the internet however - Chris Hein, Embertone, NI etc - do give you this control. So what we're probably seeing here is the "Stochastic Parrot" of ChatGBT "hallucinating". And the result is, really little more that regurgitated, undigested marketing happy talk. Albeit regurgitate, undigested marketing happy talk with a high statistical likelihood of sounding what all other marketing happy talk chatter that dominates the internet is supposed to sound like.
If you press ChatGPT further:
Q: Recommend some forum posts on working with spitfire solo strings [and give links]
You get this:
A: Here are a few forum posts that may be helpful for understanding vibrato when composing idiomatic solo string music with Spitfire Solo Strings:
1. "Tips for working with Spitfire Solo Strings" on VI-Control: https://vi-control.net/community/threads/tips-for-working-with-spitfire-solo-strings.67616/
2. "Spitfire Solo Strings - any tips or tricks?" on Gearslutz: https://www.gearslutz.com/board/music-computers/1197852-spitfire-solo-strings-any-tips-tricks.html
3. "How to make Spitfire Solo Strings sound more realistic" on Spitfire Audio's forum: https://forum.spitfireaudio.com/discussion/1609/how-to-make-spitfire-solo-strings-sound-more-realistic
3. "Spitfire Solo Strings Tips and Tricks" on The Spitfire Audio User Forums: https://forum.spitfireaudio.com/discussion/257/spitfire-solo-strings-tips-and-tricks
4. "Vibrato with Spitfire Solo Strings" on The Spitfire Audio User Forums: https://forum.spitfireaudio.com/discussion/397/vibrato-with-spitfire-solo-strings
5. "How to use vibrato on Spitfire Solo Strings" on VI-Control: https://vi-
control.net/community/threads/how-to-use-vibrato-on-spitfire-solo-strings.87213/
6. "Vibrato on Spitfire Solo Strings" on Gearslutz: https://www.gearslutz.com/board/music-computers/1250361-vibrato-spitfire-solo-strings.html
7. "Vibrato in Spitfire Solo Strings" on The Cubase Forum: https://www.steinberg.net/forums/viewtopic.php?t=105708
Which looks great ...
Except ... that so far as I can tell, none of these forum posts actually exist. They appear to be pure hallucinations.
It is interesting, however, to note just how prominently vi-c features in these hallucinations. (So: Yes. Conclusive proof that androids do indeed dream of vi-c. Happy to have settled that debate).
But again, this simply reflects the nature of the Large Language Model Transformer architectures underlying ChatGPT being - quite fundamentally - nothing more than a "stochastic parrot".
These hallucinations are really just the equivalent of chewed up and spit out marketing happy talk. Which can be, in some contexts, helpful. (Especially if you work in marketing.)
But here the parrot seems to to be completely hallucinating not only the titles of these fictional articles, but it's also inventing statistically-likely, but ultimately non existent urls. (It can also fabricate summaries, with a little prodding, which are of course, completely vacuous as well).
There's obviously a lot of tasks, including certain types of research, for which ChatGPT will be genuinely useful. But a lot of the hype around it seems to fundamentally confuse the "stochastic parrot"'s talent in regurgitating statistically likely variants of marketing happy talk for "intelligence".
One upshot is that post like this one:
FWIW, here's what ChatGPT recommends (sorry, I've been spending most of today prodding ChatGPT with all sorts of experimental requests...)
Here are some recommendations for the best basic orchestral Kontakt libraries:
- Spitfire Audio - Albion ONE: This is a comprehensive orchestral library that features a wide range of instruments, including strings, brass, woodwinds, percussion, and more. It's known for its powerful sound and versatile articulations.
- CineSamples - CineStrings CORE: This library is focused specifically on strings, offering a wide range of articulations and techniques for creating realistic and expressive string parts. It's a favorite among film composers and is known for its rich, cinematic sound.
- ProjectSAM - Symphobia: This library features a range of orchestral instruments, including strings, brass, woodwinds, and percussion, as well as a large collection of cinematic effects and textures. It's known for its powerful, epic sound and is a popular choice for film and game composers.
- EastWest - Hollywood Orchestra: This is a massive orchestral library that features multiple microphone positions and a wide range of articulations for strings, brass, woodwinds, and percussion. It's known for its high-quality sound and is used by many professional composers and producers.
- Orchestral Tools - Berlin Orchestra Inspire: This library features a range of orchestral instruments, including strings, brass, woodwinds, and percussion, as well as a large collection of cinematic effects and textures. It's known for its expressive sound and ease of use.
Needs to be viewed through the lens of the fundamental limitations of stochastic parrot technologies.
Plausibly, this kind of the regurgitation of the collective state of marketing happy talk on the internet, might, in some circumstances, offer a usefully starting point for certain types of research. But I would also argue that it is so deeply uncritical and so vulnerable to hallucination, that it there far more risk of harm that good.
The math overflow (and related) forums have explicitly banned ChatGPT content, because: "Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers." (@Mike Greene )
Temporary policy: Generative AI (e.g., ChatGPT) is banned
Moderator Note: This post has been locked to prevent comments because people have been using them for protracted debate and discussion (we've deleted over 300 comments on this post alone, not even
meta.stackoverflow.com
So I'd suggest that at the very least:
a) any content generate but stochastic parrot technologies really needs to be explicitly labeled as having as the regurgitated happy talk of a stochastic parrot technology that it is. (Which of course , that above posts does). And,
b) it needs to be read critically through the lens of the fundamental limitations of the this current class of marketing happy talk regurgitating stochastic parrot "Large Language Model" technologies. And not throughs the silly metaphors of all the internet hype, like "intelligence".
These sentiments are bouncing around on different threads. But I just though that having a concise thread on the limitations and risks of stochastic parrot technologies of ChatBPT and its ilk might be helpful.
Even just to cite, in order to avoid gum up more threads with this debate.
Last edited: