Profile cover photo
Profile photo
Graham Polley
107 followers -
Software Engineer // Google Developer Expert (cloud)
Software Engineer // Google Developer Expert (cloud)

107 followers
About
Posts

Post has attachment
Add a comment...

Post has attachment
One of the guys in our team hates crons. So, he decided to ditch them altogether and use Cloud Functions instead to trigger our Dataflow pipelines. Good call.

Post has shared content
I wanted to know how fast #BigQuery could scan, filter, aggregate, and order 1 trillion rows. So, I decided to test it.

272 seconds (40 TB).

#gde #cloudplatform #productfeedback #googlecloudplatform #bigdata
Photo
Add a comment...

Post has attachment
I wanted to know how fast #BigQuery could scan, filter, aggregate, and order 1 trillion rows. So, I decided to test it.

272 seconds (40 TB).

#gde #cloudplatform #productfeedback #googlecloudplatform #bigdata
Photo

Post has attachment
I have a friend whose hobbies include playing with the Weak Signal Propagation Reporter Network (WSPR) in his spare time.

"The WSPR is a group of amateur radio operators using K1JT's MEPT_JT digital mode to probe radio frequency propagation conditions using very low power (QRP/QRPp) transmissions".

http://dev.wsprnet.org/drupal/

My friend wanted to analyse all the public data that they have available on the web site (8+ years & ~550M rows). Over a beer or two, I told him about #BigQuery, and how it would be perfect for the analysis that he wanted to perform.

The next day I loaded it all into BigQuery for him - on my lunch break - using 7 lines of bash (wget & gsutil -m cp) and one load job from the web UI. He began analysing it the same day.

That's why I love using/sharing this tool.

p.s. I made the dataset public, so anyone can access/analyse it too: https://bigquery.cloud.google.com/dataset/dataproc-fun:wsprnet

#gde #cloudplatform #productfeedback #bigquery #wspr
Photo

Post has shared content
Add a comment...

Post has shared content
Delighted to see two of my blogs make the cut!

#gde #cloudplatform #blogpost #bigquery
2016 was a milestone year for +Google Cloud 's #BigQuery. To celebrate, we've highlighted our favorite user experiences: goo.gl/3Tu4C6

Post has shared content

Post has attachment
My latest blog post on using Google Cloud Platform to analyse Stack Overflow comment sentiment.

#gde #cloudplatform #blogpost #bigquery #ML #gcp #dataflow #stackoverflow
Add a comment...

Post has attachment
One of the guys in my team decided he wanted to check out AWS's Athena, and compare it to the mighty BigQuery.

https://shinesolutions.com/2016/12/09/will-athena-slay-bigquery/
Will Athena slay BigQuery?
Will Athena slay BigQuery?
shinesolutions.com
Wait while more posts are being loaded