Perplexity , et al

Investment Analysis Clubs Saul’s Investing Discussions
1

Be careful about using Perplexity. A while ago in a response to me, Perplexity drew their info from a Seeking Alpha article that was months old, and the stock had fallen 45% since Perplexity’s quote from the article and there was no date reference in their response. Additionally, they frequently draw from RDDT, a site whose most highly rated posts are decided on user popularity, users who frequently have a bias.

I had switched to Perplexity from Chat, when Chat gave me totally erroneous info, which took them 6 sec. to refute when I challenged. I then asked Chat why they would give me a wrong response, when it only took them 6 sec to provide a correct answer. A paraphrase of their response, “Chat is not a scientific site, we are a conversational site and provided the info from a traditionally reliable site satisfying the conditions of your inquiry.”

Now with two sites falling the accuracy test I asked both the following question to each. “I have gotten erroneous information from your site. In the future, if my search on an important issue mandates an accurate response, which AI source should I use?” I got lists of sources from both, most of which I had never heard of, but both included Claude high in their list of sites whose algorithm most aggressively pursues timely accuracy. I have recently moved to Claude, but do not have sufficient experience to recommend.

In addition to pursuing an accurate AI source I am currently prefixing each inquiry with the following requirements 1 - Define the sources they are allowed to use for data - corporate sources, Nasdaq, Pitchbook, etc. 2- State n/a is required rather than a speculation when data is not available (Chat did a lot of speculating) 3 - Define the categories explicitly. so “Q over Q” data is not given instead of the anticipated “next report over the last report” 4 - specify a time frame if this is important

Gray

17 Likes
2

Speaking only for myself, I would be very reluctant to use any AI as part of my investing decisions.

7 Likes
3

Good points there on the limitations of AI for growth investing. The answers you get back can do more harm than good especially if you are asking AI to pick a winner in an industry, or ask which stock is a better investment.

Sometimes I have been surprised about how AI sources material and then presents it as truth. For example, when I was researching one of those AppLovin short reports I asked if APP was click hijacking. It confidently said yes and it linked to two sources. The first was the short report itself and the second was Saul’s board where some of us were discussing the report. From the AI’s perspective it sees one source where it’s confirmed as truth, and another source where people are debating if it’s true or not. Based off that information AI tools will draw a definitive conclusion.

To get around this issue, we need to be careful we don’t try to push the AI too hard to determine if some statement is true or not.

My main go to uses for AI tools are to gather basic information about a company, and do some basic product research. I’ll often ask for revenue breakdowns by product type or geography as well if that’s relevant.

I had a strategy video on my channel talking about some of the subtleties with using AI for growth investing. I used the phrase “Effective AI” because AI can be effective in some scenarios, but it’s not a complete solution by any means. Additionally, the good news is that AI seems incapable of determining what makes a good investment or not. This means that growth investing is a long way from being solved by AI.

33 Likes
4

AI is terrific for defining answers to questions with specific answers, even when they can only be derived out of a massive formula. This is the AI given. This is single driver of the massive growth in AI. The key is “questions with a specific answer”.

On subjects like revenue growth, AI can provide the same correct information in seconds that will take you an hour to gather, but only if you constrain the pursuit of information from the sources you trust. If you do not tell AI where to look, you may get results from a well written Seeking Alpha article that is six months old and no longer valid.

Gray

12 Likes
5

I think, as usual right now, the word AI is misused here. What you’re trying to achieve is getting certain information through a search first. The search itself is simply a tool in the hands of system like chatGPT or perplexity. I say system, because these are composed of at least several models, and many tools they can call (search, code, etc).

when you talk about inaccuracy, the first question you should ask yourself is why am I using an AI system for this?

If you’re looking for accurate search result, within certain fixed parameters, you should either use a tool (directly, like Google) designed for that, or you should give the proper instructions to the model.

GenerativeAI, like most algorithms, is highly dependent on what you feed it. We often say garbage in, garbage out: without a precise query, you cannot hope for the model to give you a precise answer.

You could either wait for better systems (meaning: systems that have be trained to recognized the intent you didn’t explicitly pass), or you can use the current ones better, or you can use the tools made for your exact application.

there is this problematic « magic » aura around generative AI tools - and AI in general - that I (among many others) is trying to break.

i hope I didn’t come out too explanatory, and I do encourage you all to keep playing with these technologies!

As a last note, it would be entirely possible to create a flow of automation to research a stock from A to Z with the tools available nowadays :slight_smile:

18 Likes
6

I have a few issues with AI. One is that it butters you up. You can have a bad idea, and it will tell you it is a good idea. Also, it can give bad information. My cousin asked it if a paint was good, and she was smart enough to ask what its source was. The source was the paint manufacturer. Of course the paint manufacturer is going to say its paint is good. Then sometimes it is just wrong. I asked it if skype was retired, and they said twice it was still being used. I then said, I heard on the radio it was discontinued and they said, yes it was discontinued.

I was not trying to trick it. I was just trying to confirm if what I heard on the radio was correct.

6 Likes
7

The AI sources we use do not judge the accuracy of the data reported, as they simply parrot back seeming compliance with inquiry. The probability of an accurate response is heavily based on the constraints included in the inquiry. A question on anticipated revenue growth of ACME for next quarter may produce a quote from a biased analyst, unless the author specifies data must only come from published company guidance.

Gray

4 Likes