The odd things you encounter reading novels

The odd things you encounter reading novels: I’m reading a novel called Mr. Penumbra’s 24-Hour Bookstore which is a quirky, light, magical, mystery, apparently bordering on science fiction, and perhaps something of a cult-book for some people.

Kat, who works at Google, and Lenny, the narrator, who works at Mr. Penumbra’s 24-Hour Bookstore, are in Kat’s bedroom. They want to put a thick hand-written ledger on the computer in a hurry:

She’s sitting on my lap, leaning into her MacBook. She’s explaining OCR, the process by which a computer transforms swoops of ink and streaks of graphite into characters it can comprehend, like K and A and T.

“It’s not trivial,” she says. “That was a big book, and bad handwriting.” But Kat has a plan. “It would take my computer all night to process all those pages,” she says. “But we’re impatient, right?” She’s typing at warp speed, composing long commands I do not understand. Yes, we definitely are impatient.

“So we’ll get hundreds of machines to do it all at once. We’ll use Hapdoop.”

“Hadoop?”

“Everybody uses it. Google, Facebook, the NSA. It’s software – it breaks a big job into lots of tiny pieces and spreads them out to lots of different computers at the same time.”

Hadoop! I love the sound of it. Kat, you and I will have a son, and we will name him Hadoop, and he will be a great warrior, a king!

Best description of Hadoop I ever read!
Saul

21 Likes

“Everybody uses it. Google, Facebook, the NSA. It’s software – it breaks a big job into lots of tiny pieces and spreads them out to lots of different computers at the same time.”

So how do you input the big book with bad handwriting so Hadoop can work its magic? Do they have a high-speed scanner that slits the binding or turns the pages rapidly so it can be scanned?

1 Like

So how do you input the big book with bad handwriting so Hadoop can work its magic? Do they have a high-speed scanner that slits the binding or turns the pages rapidly so it can be scanned?

Apparently they solved this problem in Germany. I read that the western government or someone in the west wanted to read the document the east Germans has shredded when the Berlin Wall fell. I found it interesting enough to read but not enough to bookmark so I don’t have any links.

But Google is amazing!

2003: http://news.bbc.co.uk/2/hi/europe/3279055.stm

2007: http://www.unshredder.com/nov-2007-computers-to-reassemble-s…

2010: http://content.time.com/time/business/article/0,8599,1983287…

2012: http://www.theaustralian.com.au/news/world/computer-ready-to…

Not quite warp speed…

Denny Schlesinger

1 Like

Again, I’m not a techie, but Google actually has machines that turn the pages and scan books. Been doing it for years. But I was amazed to hear Hadoop referenced in a novel!
Saul

1 Like

“Apparently they solved this problem in Germany.”

Denny,

It is the Fraunhofer IPK, Berlin, that took on the task of virtual reconstruction the hand-shredded Stasi files, initially only 400 of the over 15,500 sacks of ripped documents. The information is from 2013: https://www.ipk.fraunhofer.de/en/divisions/automation-techno….

I lived in Germany when they started the reconstruction project and the arguments pro and con. This video by the Fraunhofer IPK shows the entire process of sorting, scanning (they had to build their own scanner), reconstituting pages (they wrote their own software), etc. Other governments, agencies, and institutions, s. a. archeology, have shown great interest in their endeavor and its possibilities. Here is the link to Fraunhofer IPK’s video, also produced in 2013 and for anyone interested in this kind of thing, well worth the 14 minutes:

https://www.youtube.com/watch?v=FQdKf1H-iXg.

im

4 Likes

Yes. Quite an amazing project!

Denny Schlesinger

Hadoop has a better competitor:

If the novel were re-written today, she would have used one server with an NVDA Volta chip & CUDA language. Instead of using the elctricity expended in one hundred old line servers connected with Hadoop, the one new server with “Nvidia Inside” and CUDA ecosystem would do it all, and faster. Think A/I. Think the brains of autonomous cars needing split second decision making while the car moves through space and time.

Hadoop? Man, that’ so 2016.

7 Likes

Hadoop has a better competitor:

If the novel were re-written today, she would have used one server with an NVDA Volta chip & CUDA language. Instead of using the elctricity expended in one hundred old line servers connected with Hadoop, the one new server with “Nvidia Inside” and CUDA ecosystem would do it all, and faster. Think A/I. Think the brains of autonomous cars needing split second decision making while the car moves through space and time.

Hadoop? Man, that’ so 2016.

Hi RockOYates, You’re right! The novel was written in 2014. Things become out of date fast nowadays!
Sau

1 Like

Saul - thanks for the recommendation, I saw the novel when it came out but had forgotten about it. I started it last night, enjoying it already.

best,

Naj