OT: Data Science Question

I know there are some really smart data guys here. This is cross posted to “Help with this Stupid Computer”

https://discussion.fool.com/data-about-pressure-changes-34164677…

My wife suffers from
migraines. It seems the barometric pressure changes are one of the triggers. (From what I understand there are many and the interactions are complex.)

I might have the opportunity to move to the U.S. Virgin Islands or Half Moon Bay in California. Both of these seem to be places with less barametic pressure changes than in the Mid West.

So I found this list of sites with a history of barometric pressure changes.

All I have to do is down load the 10 Gigabits. Clean it and load it into SQL.

R or Python should work.

ftp://ftp.ncdc.noaa.gov/pub/data/noaa/1901/029070-99999-1901

I have a little
experience with data science. I have taken some
courses in R and Python, and SQL, but I do not have the experience of actually cleaning the data and then queerying it. (I am a technician and have never completed the calculus and other advanced math to become a data scientist, and had to delay my programming classes for courses in networking for the last couple of years)

I don’t know if Excel
can handle 10 gigabits of data.

Ideas on scripting and parsing?

My computing power is low. I may need to rebuild my desktop. I can blow a computer out and load Linux. Or, I have access to a retired server. I
was thinking of loading VMware on it anyway. Or I can just buy an awesome
mother board and SSD drive and graphics card and a ton of ram. Or I can use cloud computing. I have no
experience in cloud computing and the only training was a VMware class.

Ideas on compute needed?

If I get far enough into this I will try to correlate the data with migraine prescriptions filled and see if the pressures actually do correlate with pressure changes and see if the
move would be worth while.

Cheers
Qazulight

I don’t know if Excel
can handle 10 gigabits of data.

The file you linked is less than 150K. That file can be prepared and processed with a text editor and Excel. KISS! :innocent:

Denny Schlesinger

Please do not reply in this thread. If you want to reply to qazu please check reply to author and uncheck Post reply on boards.

Very OT. Thanks everyone.

6 Likes

Thanks Captain,

I posted the wrong link. I need all the files from 1901 to 2018 and the supporting documents also.

I posted the link to only 1901.

Sorry

Or follow the link to the computer board. I think this would be useful public discussion on the other board.

Thanks
Qazulight