Profile

Cover photo
Brian O'Neill
303 followers|360,356 views
AboutPostsPhotosYouTube

Stream

Brian O'Neill

Shared publicly  - 
 
Monetate: Philly's Best Technology Work Culture
Since I moved back to Philly from Silicon Valley, I've had the following on my resume: Goal: To bring the creative, inventive spirit of Silicon Valley to the Philadelphia region. Monetate is doing exactly this.   It is the first company for which I've worke...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Monetate open-sources Koupler: a versatile interface to Kinesis!
I'm happy to announce that Monetate has open-sourced Koupler, a versatile interface for Kinesis.  We took the best practices outlined by Amazon  and codified them. Hopefully, this will be the first of many contributions back to the community. For the full s...
1
Eddie C's profile photoBrian O'Neill's profile photo
2 comments
 
On the consumer side, we're actually using Spark.
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Using Gradle with AWS and S3 (w/ DefaultAWSCredentialsProviderChain and InstanceProfileCredentialsProvider (FTW!))
We recently switched over to gradle as our build mechanism.  As part of that switchover, we wanted to be able to build w/o external dependencies (e.g. Maven Central).  We tossed around the idea of Artifactory, but in the end we decided to keep it simple and...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
A quick proof that Black is not White (to save all of mankind)
In the Hitchhiker's guide to the galaxy, there is a proof about the non-existence of God.  The reasoning roughly mimics that of the Ontological argument for God . The passage has stuck with me: "I refuse to prove that I exist," says God, "for proof denies f...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Destinations for syslog-ng : UDP + Program (Java)
This is just a quick mental note because I keep losing them... Here are two quick gists that show how to configure destinations within syslog-ng. UDP Destination This destination will spew UDP datagram packets: destination udp_spew { udp("localhost"
port...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Druid : Vagrant Up (and Tranquility!)
We've been shown the light.   After a great meeting with Fangjin and Giam , we are headed down the Tranquility route for real-time ingestion from Kinesis to Druid.  For me, that means getting comfortable with the expanded set of node types in the Druid clus...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
PySpark on Amazon EMR w/ Kinesis
Spark is fantastic.  And its streaming framework has proven to be a perfect fit, functioning as the real-time leg of a lambda architecture. In our initial use of Spark, we decided to go with Java, since Spark runs native on the JVM. (and sorry Scala, still ...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Using Squid as an HTTP Proxy via SSH (to fetch remotely from Amazon yum repos)
We've been playing around with vagrant for local development.   When combined with Ansible, the pair allows you to recreate complex systems locally with high fidelity to your deployment environment.   Through magic voodoo (kudos to @jjpersch and @kmolendyke...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Diagnosing memory leaks in Java
Every time I suspect a memory leak, I have to go dig up these commands. So, here they are for posterity's sake: First, I use the following command to monitor the process over time: while ( sleep 1 ) ; do ps -p $PID -o %cpu,%mem,rss ; done (and/or New Relic...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Reverse Engineering Kinesis Aggregation and Sharding (and the magic "a" partitionKey)
I love open source.  I love being able to take things apart to understand them.  And for that reason, I'd like to thank Amazon for open sourcing Kinesis Consumer Library  and Kinesis Producer Library .  It allowed us to get to the bottom of a couple of stra...
I love open source. I love being able to take things apart to understand them. And for that reason, I'd like to thank Amazon for open sourcing Kinesis Consumer Library and Kinesis Producer Library. It allowed us to get to ...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Integrating Syslog w/ Kinesis : Anticipating use of the Firehose
On the heals of the Kinesis Firehose announcement , more people are going to be looking to integrate Kinesis with logging systems. (to expedite/simplify the ingestion of logs into S3 and Redshift)  Here is one take on solving that problem that integrates sy...
On the heals of the Kinesis Firehose announcement, more people are going to be looking to integrate Kinesis with logging systems. (to expedite/simplify the ingestion of logs into S3 and Redshift)  Here is one take on solving ...
1
Add a comment...

Brian O'Neill

Shared publicly  - 
 
Getting started w/ Python Kinesis Consumer Library (KCL) (via IPython Notebook)
We are racing ahead with Kinesis.  With streaming events in hand, we need to get our data scientists tapped in. One problem, the KCL and KPL are heavily focused on Java, but our data scientists (and the rest of our organization) love Python. =) We use IPyth...
1
Add a comment...
Story
Introduction
Husband, Hacker, Hiker, and Kayaker. Fisherman and Father. 
Big Data Believer, Innovator, and Distributed Computing Fanatic.
DataStax MVP & Rebel Elite, InfoWorld's Technology Innovation Award Winner

Brian is Principal Architect at Monetate, Inc, where he is part of a rock-star team building out a cloud-based platform that leverages context and real-time analytics to personalize the interaction between marketers and consumers across all media channels.

Previously Brian was CTO at Health Market Science. (acquired by LexisNexis)
Links
Contributor to
Basic Information
Gender
Male