Hello and thanks!

Stocks A to Z
1

Hello everyone

I’ve been passively following the Berkshire, Mechanical Investing, Falling Knives, and METAR boards for well over 10 years, but I only recently got around to actually setting up an account here.

And I’ve been meaning to post… I just hadn’t quite gotten around to it yet. Oh dear.

Before the boards begin their transition, I want to thank everyone who posts on TMF for sharing so many interesting ideas and thoughts and articles over the years.

I’m certain I’ve learned something from every person here. (Though, 1st place for sharing of great ideas surely has to go to mungofitch.)

I am also grateful to the owners & staff of TMF for hosting this wonderful oasis, deep in the desert of investing, for so long.

About preserving the existing historical content of the boards long term.

A noble goal. Many of the ideas that can be found here in the deep history of the boards - especially on the BRK board and MI board (and Saul’s board too, if I were interested in hypergrowth investing) - or the METAR board for ‘how it seemed at the time’ - and many more - these ideas and fragments of finance history, simply can’t be found anywhere else on the internet, or in journals or in books.

When TMF UK was closing down its discussion boards many years ago, I contacted a member of the IT staff and asked them if they could generate an SQL dump - a kind of structured, compact ‘database image’ file - with a selection of my favourite posts, and they very kindly helped with that.

Perhaps the staff of TMF USA would be willing to generate a public or private SQL dump of all the boards, or at least the most popular boards, to ensure that the history of TMF will never be lost. An SQL dump is a lot quicker and simpler to produce today and re-host later (if ever needed) than having random individuals manually spidering pages. An SQL dump is also more likely to be complete, and to require very little extra work to re-host it, if the current pages ever fade away.

Anyway I’m sad to see the boards changing, because I think the look & feel & culture of the TMF boards, today, is just perfect. But I suppose I’m not the one who has to keep things running behind the scenes. And I should be glad anything exists at all!

Thanks again to all the active posters for so many years of wonderful posts here.

I look forward to reading many more great posts on the next iteration of the boards, too.

(And maybe my second post will be more promptly written than this first one.)

LM

p.s. A request to Jim, and on the topic of archives.

Would it be possible to read through an archive of the newsletters you produced - “The Informed Hunch”?

29 Likes
2

And maybe my second post will be more promptly written than this first one

Hopefully!

What a nice constructive change this post is to mine and other posters usual chattering, mostly going in circles. Fresh blood!

I hope you forwarded your suggestion to TMF.

3 Likes
3

Would it be possible to read through an archive of the newsletters you produced - “The Informed Hunch”?

Not really.
They were aimed at a very small private audience, and I probably said some things with a little more…frankness…than I would feel comfortable with in a public forum.
Not the politically incorrect or foul mouthed, but it sometimes included my portfolio holdings.
Only a few specific items were ever posted, like this one https://discussion.fool.com/getting-away-from-the-bear-27035352…

Besides, most of the comments were timely at the time, but of relatively little use years later.
Does anybody care that in April 2005, with the price around $80000, I predicted that Berkshire would trade at $134000 by the end of 2007?
(It happened in November 2007, so I was just under the wire!)

Besides, I’m embarrassed at the many bad predictions : )

Jim

9 Likes
4

Hello said2,

It’s a pleasure to correspond with you after many years of enjoying your thoughts & writing.

I hope you forwarded your suggestion to TMF.

I would rather one - or ideally several - of the TMF regulars (mungofitch, yourself, WendyBG, OrmontUS, Saul, etc) put over this idea to TMF to secure the future of all this wonderful writing, no matter what happens to TMF’s ability to sustain it online themselves.

If I forward the idea to them myself, they would have absolutely no idea who I was and there would be no reserve of goodwill and trust to draw upon.

A total stranger popping up and saying ‘excuse me lads, can I have your entire database?’ is usually not well received by DB admins.

Ideally, if a few of senior people ‘concerned for the future of the history’ could forward a link to this post and my other one, I think that would contain much of the information needed for IT & management at TMF to make a decision about the idea, and whether/how/where to do it, I think.

Time is of the essence when web platforms enter a sunset period, so if you see this idea as something worth investigating, right now would be the best time for you (dear reader), to pass it to them or ask for it directly.

I should probably add some brief further remarks to my comment about an SQL dump and archiving the site as it stands, as well as comments about valuable historical internet datasets, open data in general.

  • My background includes years working as chief engineer & lead database admin responsible for operating large national/government databases for a developed country, which had some good institutional open data and open source policies. Based on my background and experience, I think the sqldump idea in my previous post is quite likely to be practical and straightforward (and valuable for ‘the future internet’, and perhaps especially valuable for academics in the field of finance), so it’s really up to the decision makers involved, whether it’s a thing they want to do. It’s at least practical and easy, and avoids having numerous people crawling the site with badly written or aggressively configured site spiders over the next month to recover ad hoc extracts of the past.

  • The work involved to produce an sqldump is minimal, normally just a single command for most database servers. It can be written in minutes and would take perhaps 1-5 hours to run (I’m guessing TMF’s database is about 50-200GB in size?). It would look a bit like this:

(sqldumpcommand) (everything including schema) (excluding system tables and user data - email, name, password)

and perhaps a SQL command beforehand to subtract out ‘deleted but not forgotten posts’ forever, or shift them to a private, non-archived table.

  • Making a database sqldump is likely to already be part of the normal backup process for the IT team so the added workload should be fairly tiny.

  • A key question is: where does the sql dump file go? And who has access? Remember, it will be big - 50-200GB perhaps.

  • I think Wikipedia have the best open data model here, and are a great point of comparison, as Wikipedia is similar to the TMF Discussion Boards - both are essentially a large, long-term collection of quality amateur writing from a variety of internet people on many interesting topics.

  • Wikipedia’s view is: this collection of writing was produced by all the internet, so it belongs to all the internet forever, and anyone can have their own copy of all the writing forever. Not only as webpages, but also as a structured database file.

e.g. Read through: https://dumps.wikimedia.org/

  • Another idea (in addition to the above) is to ask The Internet Archive if they would be willing to host an sqldump and also web-based backup of the site. Or to simply do it directly via their tools. They offer the ‘wayback machine’ as a general web service, but they also host e.g. specific dump & archive files that bring together neatly a large volume of important internet data & history and can be downloaded/backed up by data archivists, academics, and enthusiasts, as well as those who are simply sentimental about an era of their lives on the internet.

  • IMHO adding the whole dataset as an SQL dump in their archives would be extremely valuable for future finance amateurs, professionals, academics, historians etc.

e.g. https://help.archive.org/help/uploading-tips/

e.g. https://www.sucho.org/ia-bulk-upload

LM

12 Likes
5

LM,

Totally agree and thank you for your passion and making the point to TMF so clearly on behalf of us ALL who have been educated, enriched and entertained from these wonderful and thoughtful posts from so many of all ages, nationalities and perspectives.

2 Likes
6

Hello Jim,

As I just said to said2, (pardon the pun), it’s wonderful to correspond after so many years of enjoying your thoughts and writing.

Not really. They were aimed at a very small private audience

Thanks for considering my suggestion. I respect your decision. I totally accept that private correspondence with friends, vs public correspondence for internet crowds, are quite different in nature.

Besides, I’m embarrassed at the many bad predictions : )

I don’t think you should feel embarrassed about some past predictions not being great. It might feel far worse if they HAD all turned out spectacularly well but you hadn’t bet the house on all of them…

I think too, if anyone’s past predictions don’t seem naive & silly to their future self, then it’s a clear sign that a person hasn’t grown in wisdom over the years. The price of long-lasting wisdom can be paid either in dollars, pounds, or social embarrassment.

The only way to really put one’s thoughts to the test is to put something at stake - whether it’s money or reputation among friends. And without ever putting ourselves to the test, how can we improve as investors?

‘Bad predictions’ are where both the volume and detail of life’s learning occurs, especially when they are revisited and reflected upon later, to analyze both the mistake, and the form of thinking & assumptions & models that lead to the mistake.

Things like newsletters (or similar public ‘position essays’) are best of all, because it is not merely a chance comment at that point that turned out well or badly - it’s a structured, contemplated block of thought and conclusion.

Like the difference between playing a round of sport on a fun/casual basis, or a competitive basis, among friends.

Certainly, in Buffett’s essays, the parts I have learned the most from are those where he describes his past folly, and WHY it was clearly folly.

I think it’s wonderful and educational to chuckle at our own historical ‘oops’. Of course, it’s easy for me to say that, as someone who only began posting ideas here just hours ago.

LM

5 Likes
7

One extra note. I saw somewhere, people asking ‘how will links be useful when the old URLs no longer work’. This is a long-solved problem if the site is archived in a sql dump or set of spidered files. Something called a ‘regular expression’ can be run to remap http://oldaddress/oldpath/messagename.html to http://newaddress/newpath/messagename.html. It’s basically an automated find/replace like the one you see in Microsoft Word, but with more programmability. Pretty much a one line command.

8

I actually wrote a script on the weekend to crawl the old BRK board and save all the raw posts in html for posterity. So theoretically I have them all as long as I didn’t have a bug.

The only thing though is that I don’t know that I (or anyone) would be allowed to publicly host them. Hopefully there’s no need if they can just migrate them all over to the new platform.

2 Likes
9

I don’t think this platform will have anything like the degree of discoverability and searchability the old platform offered. Hopefully I’m wrong.

10

Click the magnifying glass in the upper right, then click three lines to open advanced search, it’s much better than the old one.

As for finding interesting posts, the Everything or Latest views let you see all posts, not just ones in the categories you track - if you want to look for other things. Plus, there is the Top post feature on the home page which brings interesting posts to the top.

3 Likes
11

I have no doubt the search is better (I depended on google for that in the past with site:discussion.fool.com).

The issue I’m finding - even as a technologist with > 3 decades in programming and IT - is that the interface is visually overwhelming and complex - lots of buttons, icons, notifications, irrelevant information etc.

What I enjoyed in the past was the fact there were just … loads and loads of ordinary text links to click on. With small fonts meaning high density of choices. Whether boards or posts. Simple.

Just now for example, in my field of vision I have ‘bookmark’, ‘last visit’ ‘21m’, ‘19m ago’ ‘9/10’ ‘Sep 27’, ‘You will receive notifications…’ various (1), multiple ‘unread’, etc - huge amounts of icons, colour etc distracting me / demanding my attention. I can hardly count the total number of icons & ‘facts’ on screen in front of me. Perhaps I’ll look around and see how much of it I can disable. Front page is just waves of icons.

(If ‘high density text’ and ‘too many things’ seems like a contradiction, it’s more that the irregular placement, colour, function, factoids etc that are troubling. 20 lines of text in a row in a box are easier to cope with).

This interface is a little bit hard for ADHD sufferers.

However on the plus side - again as a technologist - well done on the platform migration, it seems to have gone very smoothly.

11 Likes
12

I would support your idea of downloading the previous messages into a searchable database

I’m pretty sure I can get WendyBG (I’m meeting with her this weekend) on board as well.

Jeff

2 Likes
13

It’s possible to manually spider the site and extract the text & names & recs out with a little bit of parsing, and insert that into a database, but I don’t think it’s the right way to do it - it’s a circuitous solution. The best solution when large volumes of data are involved is usually cooperative and just sql dump straight out.

Hopefully the staff of TMF will be receptive to such an approach if approached by yourself & Wendy but I would emphasize ‘as soon as possible’ is best because sunset periods on tech platforms often pass faster than you’d think.

The other advantage of a DB/dump solution is that SQL is a very stable format for the long term, and can be easily adapted to later web interfaces, can be searched hyper-efficiently, and so on.

It is probably worth pointing out to TMF management that the commercial value of the historical message board is likely close to zero, because the old boards won’t be used as a platform any more, and many of the authors & readers of the time have moved on or passed away. It’s primarily sentimental, historical and educational value at this stage.

3 Likes
14

Just posted this on another thread but copying here.

1 - usernames would need to be included somehow, but not other data, to attribute correctly to each person

2 - A full DB dump might not be needed -

"Ultimately, an SQL dump for historical preservation doesn’t have to be a fully working and coherent database from the get go, e.g. the existing database might have data sanity checks, triggers etc - it simply needs to contain enough of the data that a working database can be set up from it later on.

Even a crude materialised view of existing tables… i.e. something along the lines of

Select (username, recs, date_posted, board_posted_onto, subject, message text) from (join lots of things where …=…) into (a new table called) historical_archive.

This would be a dumbed down version of the database, denormalised etc into a single giant table, simply a collection of the written messages, each alongside the same info you’d find on a web page printout - but likely 100% adequate for the task of preserving history, and notably, complete."

2 Likes
15

Yes, it is. The key for me has been learning what to ignore (a lot), and what to pay attention to (not that much). I’m still finding my way around, but I am already working with it pretty effectively.

I posted something aimed at following a specific board here: Found the new Board - #7 by RHinCT

For following everything I followed before, I had to find the Tracked item under Community at the top of the left sidebar. For me it was invisible, hidden within Community, until I clicked on Community. Tracked limits the view to only the boards that were my Favorites on the old system.

3 Likes