I recently attend the AWS re:invent conference in Las Vegas. It has become an annual trip for many but this was a first for me. My goal was to get exposure to their service offerings and see what the buzz is about. Up until the conference I had zero hands-on experience, but rather heard from many customers of their desire to move to the cloud. I’ve spent lots of time working in clustered environments (e.g. Hadoop) but never within AWS.

I’ll first say that it is big – roughly 19,000 people attended. It was a very full 2-day event with additional content on both ends. I sprung for a bootcamp (Building a Big Data Platform on AWS) before the conference so I could get a bit of hands-on time so I didn’t go in totally blind. Turns out this 100-person training class was some of the most useful time I had. It was AWS at the ground level with a good mix of lecture and hands-on time. It was broken into two sections:

  • System Architecture – compare/contrast to Hadoop ecosystem, tools overview (S3, EMR, Kinesis, DynamoDB, Redshift), labs
  • Redshift – use cases, compression, query performance/optimization, integrations, more labs

The next two days started with a number of keynote speakers followed by an incredible amount of breakout sessions (270+) depending on what you were looking for. They also had hands-on labs where you could work with different tools at your own pace with AWS consultants to guide you along the way. The breakout sessions covered everything from real world implementations to tool deep-dives to machine learning and DevOps. Without going too much into the content of these (many of which are available online) I wanted to convey my main takeaways from the conference as a whole.

  • Overall I found it to be exactly what I was looking for in an introduction to the environment. There was plenty of hype during the keynotes but lots of applicable real-world examples and experiences displayed in the breakouts.
  • The bootcamp was by far my best time spent while attending. It was a smaller atmosphere designed for those with minimal exposure to AWS. I took away the content and labs and am free to further explore the content covered at my own pace. All of the tools covered were foundational, key areas that big data systems would require.
  • The conference provided a base that I can now build upon at my own pace. I can speak the language and relate AWS tools to those I’m used to. I can see how the systems I’m familiar with could be migrated and improved upon using AWS.
  • Useful links:

Overall I came away very impressed with the AWS offering.  They have conveniently packaged the creation, management and integration of big data components on their own hardware. Before attending, my experience has only been with customers that procured, installed, managed and operated their own hardware – which is a huge undertaking. From a high-level, the underlying technology AWS provides is exactly the same as a self-managed rack, but amazingly easier to manage. Need more hardware – spin it up. Need bigger boxes – they have them. Need storage – pick your flavor. I look forward to more hands-on work in the future.