DevOps so easy a 5 year old can do it
Last weekend, I was worried about the operational state of our analytics pipeline at work. For the uninitiated, its basically a series of ‘jobs’ that process records from each page view on our customers’ websites and tabulates all sorts of reports from that data. It runs 24 hours a day. It usually runs smoothly but occasionally it’ll have a problem in production that we need to solve. And since I’m the earliest riser on the team that is responsible for it, when I wake up, I check on its state. Lets level set though — I’m a product manager, not an engineer. I’m not doing anything heroic here. Pretty much the best thing I can do at 6AM when I wake up and see that the processing is heading in the wrong direction, is say “crap, someone smarter than me needs to wake up to fix this”.
So part of my responsibility is monitoring the well-being of our operations. But another part of my responsibility at 6AM is my kids. I can happily live on less than 7 hours of sleep which means I’m ready to rock and roll with the them on the weekends when the kids are up at 6 AM. So last weekend, I find myself downstairs with my 5 year old. I’m moderately concerned with the state of the pipeline in our production environment. I have my laptop out and am checking out our monitoring tools. But my 5 year old son is, you know, 5. And he’s hungry because its been something like 12 hours since the last time he ate. And expecting pancakes.
So I do what every geek dad would do. I give a 30 second explanation of what a flow diagram is and the meaning of the colors on the graph of the flow diagram in question. And then I put him to work. Because there are pancakes to be made. The following ensues:
I don’t generally post things publicly with my kids, but I thought this was cool (and I figure he’s facing away from the camera).
Every production process should be this easy to monitor and watch. Of course my 5 year old (or me) can’t do anything if those boxes turn the wrong color, but . . . my 5 year old understands what’s happening and is pretty close to being able to identify if a Bad Thing is happening. The data visualization is simple and it works. Its intuitive. The design is pretty rudimentary in terms of styling but the visualization of the data delivers a clear, easily understood story. Not too much to juggle and a mere mortal gets it quickly. It solves a key problem – informing a broad set of mortals in our organization how key operations are executing. The more people in your organization who understand the state of your most complex operations, the better you can all do your jobs. I love love love it when I stroll into HubSpot’s support area and see folks looking at this and other operations graphs. Your entire operation works more efficiently. Support teams can respond in real time to customer inquiries without “asking the engineers” and engineers are focused on either moving your product forward or solving the devOps problem du jour without major distraction.
Anyway, thought this was too good to not share. Enjoy.
Running my first half-marathon
I try to run frequently for exercise because I enjoy it. It helps me numerous ways. But until this summer, I’d never really run a distance longer than 6 miles. This summer, my friend John challenged me to try to run a half marathon in my hometown. At first, I was reluctant, figuring the pounding on my wimpy, skinny legs would likely injure me. But I relented, registered for the Chilly Half-Marathon in my hometown and gave it a shot. I’m glad I did. Some thoughts on the process:
1) It really gave me an extra dose of motivation to run at least four times a week. That was worth it, in and of itself. The prospect of failing at the race (or bailing in advance) kept me running consistently.
2) The mileage didn’t really cause any injuries. I’ve been focusing on shortening my stride and moving my foot strike from my heel. I think that paid off.
3) I found I really really loved the longer distances. My standard runs are all about four miles. When I trained for the half-marathon, I did a weekly long run on Sunday mornings. Gradually these long runs built up from six miles up to 12. For some reason, breaking through the ten mile barrier seemed like an imposing task. Once I did it, I found I liked those 10+ mile runs the best. Time floated by from miles 6 – 10 and my brain seemed to slow down. It was my favorite part of the training, hands down. I found myself really looking forward to my long training runs, days in advance.
4) The drama for me came when I was diagnosed with walking pneumonia 6 days before the race. Race day was a Sunday. It threw off my last week of training badly (missed my last long run the Sunday prior to the race), I didn’t run Monday through Wednesday of that week either. I was on antibiotics. I felt well enough to run on Thursday and Friday and ran pretty well those days, felt fine.
I asked my doctor if I was stupid to run a half-marathon 6 days after being diagnosed. He dryly said “I wouldn’t say ‘stupid’ . . .” and proceeded to tell me that if I felt ok, it was all right by him to give it a shot. Basically, he told me to listen to my body and channeling Brian Halligan to “use good judgement”.
So I gave it a shot on race day.
I went out cautious from the start, in part because it was my first half-marathon but mostly because I was nervous about the pneumonia. But when I got to mile 3 and the timer called out my split as close to 28 minutes, I said “screw that, I want a better time than that” and turn things up a few notches.
I don’t regret it. I finished in 1 hour and 48 minutes. About an 8:15 pace and the pneumonia didn’t really impact me while running the race. I was really happy with the result — my training runs had been about 15 seconds per mile slower on terrain not quite as hilly as the race course.
I think the half marathon set back my recovery from pneumonia a few days (flying out to Denver to be at 5000 feet for 3 days didn’t help either), but I’m no worse for the wear now.
In short: the goal of running a race provided more motivation; the longer distances were very relaxing; don’t get pneumonia the week before the race.
What’s next? I’m thinking I’ll give another half-marathon a shot — maybe something in the spring like this half-marathon near my parents’ house in Gloucester. Anyone want to join?
3 Most Useful Blog Posts Ever
What blog post has been most useful to you?
I find myself constantly referencing useful blog posts in my life as a product manager. Not only that, I tend to go back to a few key posts, by a few key influential bloggers. I thought I would recount the 3 most useful blog posts I’ve ever read, as determined scientifically with complete and absolute statistical certainty.
1) I should start with Joel Spolsky since Joel on Software was probably the first blog (in the dark ages of blogs) that I ever read. I started reading Joel on Software when I was working at Kenan Systems as a release project manager for the software product. Aidan, one of the engineers suggested I read a website that he enjoyed. In no time, I was devouring every article in the archive. This was probably 2001 or 2002.
Joel wrote a ton of great posts — if you’re in the software building business, you should certainly go read his greatest hits, at a minimum. But the one that stuck with me is the one about rewriting software products. It probably sticks with me because of the firmness of the advice — rewriting your code from scratch, Joel writes, is “single worst strategic mistake” a software company can make. Not only that, its advice I find myself calling upon, every day. At any company, there’s production hardened code. And it has flaws. And its in outdated, hard to use languages. But “when you throw away code and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work.” Its a calculus I find myself walking through very often these days. Its timeless advice.
2) I read Fred Wilson every day for a very long time. Much of the time I was at IBM. It was like he was writing about an alternative reality to my every day life in enterprise software.
But the moment I joined HubSpot, his knowledge skyrocketed in important to me. The one post I find myself repeatedly referencing is Fred’s 10 Golden Principles of Successful Web Apps. Especially influential has been his point about “instant utility.” Maybe this resonated with me because “utility” is probably the only concept from philosophy class I remember. But more likely because its the absolute key to many of the most successful elements of the HubSpot portfolio. If your website or product doesn’t quickly deliver something useful to the customer, then you’re doing something wrong. The closer to the consumer market you get, the more important this advice is. I bust out this blog post at least once a month in product design discussions. I think the HubSpot wiki must have at least 2 or 3 comments of mine linking to it.
3) Last, but certainly not least, is Marc Andreesen. Andreesen is like the EF Hutton of the software world for me. When Marc Andreesen talks, I listen. When he was actively blogging on his personal site, the signal to noise ratio in his blog was off the chart. His writing and perspective was incredible. I vividly recall reading his take on the financial meltdown — and how he basically called it blow by blow.
But his post that has stuck with me and helped me in my career was his writeup on personal productivity. A few days ago I wrote about one element of this post — not keeping a schedule. More so than that advice, I’ve consistently tried to adopt almost every day since is the suggestion around keeping a 3×5 card with my daily to-do’s. Andreesen writes:
I sit down at my desk before I go to sleep, pull up my Todo List [...] and pick out the 3 to 5 things I am going to get done tomorrow. I write those things on a fresh 3×5 card, lay the card out with my card keys, and go to bed. Then, the next day, I try like hell to get just those things done. If I do, it was a successful day.
As I read elsewhere, keeping the list for a single day keeps me focused and the satisfaction of physically crossing off the completed items is immensely satisfying. Andreesen’s other suggestion about “productive procrastination” is also golden and recommended. All that blog reading when I should have otherwise done work has paid off.
What blog post has been most useful to you?
Plan for a Flood, Get a Tree on Your Roof
Three things I learned from Hurricane Irene:
1. If you have a sump pump in your home and you’re worried about the power going out, the day before a hurricane is not the time to try to go buy a generator.
That said, the ‘day before panic purchase’ tact had worked in the past for me with a snow blower. I found that Home Depot and Lowes tend to keep limited stock in their store for big ticket items like that. Presumably Home Depot and Lowes are run by MBA’s, well schooled in the latest supply chain theories. Sears, on the other hand, seemed to have a massive on-site stock of snow blowers. It didn’t work this time with the generator. Good thing I didn’t need it because . . .
2. If you plan for a flood, mother nature will laugh her ass off at you and put a tree on your roof
(Worry not, damage is minor)
3. Its good to have an old classmate who’s switched careers and become a tree specialist. Now I just need a friend who became a roofer . . .
Changing Jobs, Changing Control
I came to HubSpot about 14 months ago. It was a big change for me professionally and personally. At the time of the change, I wrote a bunch of posts on this blog recounting the job search process and my adjustment to life outside of IBM and to a growing startup. With a full year under my belt, I gave some thought to the biggest changes and adjustments. The conclusion I came to was that I now have so much more control over the course of each and every day – and its incredibly liberating and satisfying. Its not that big bad controlling IBM didn’t give me autonomy — it did. It gave me a remarkable amount of control over big decisions for the products I worked upon. This feeling of control and liberty came more through the mechanics of no longer needing to coordinate and collaborate with the web of IBM colleagues and peers. Here are two examples
1. Meetings.
My volume of meetings went from 6 – 9 hours of daily meetings (primarily conference calls) at IBM to . . . well, most typically 15 minutes a day. Maximum at HubSpot, I have 3 meetings in a day. Whereas at IBM there are all sorts of status meetings and coordination meetings and pre-meetings and review meetings, at HubSpot, I have none of them. This is a good thing not because meetings are de facto “bad”. Its more an indicator that the nature of my job changed entirely. I went from a place where I was keeping a manager’s schedule to a place where I was keeping a maker’s schedule. There is now at HubSpot a clear expectation and an actual opportunity through the course of the day for me to actually . . . do stuff. I actively contribute to the development of the product. There’s a 15 minute status meeting each day with my team, as part of Scrum, which serves a clear purpose. Its brief and done standing up. Other than that, its not uncommon for me to go the rest of the day meetingless — and its an absolute joy.
It is a joy because I truly feel productive. I feel like I’m making a difference at HubSpot. This really came home for me today when I re-skimmed Marc Andreesen’s Guide to Personal Productivity. Not only do I try to follow his index card trick, but I fell into his suggestion to not keep a schedule. Now I don’t do either of these things completely, but I’m almost entirely in control of my day now. I can focus on the things I think are most important for my job and for HubSpot. Not what the other people need and want. And it is incredibly satisfying.
2. Email.
With a modern startup, I have left the tyranny of the Lotus Notes/Microsoft Outlook inbox behind. Once I adjusted to the gmail threading approach to the inbox, my efficiency with email skyrocketed. Whereas previously I had frequently been declaring email bankruptcy, now I’m in the free and clear with my inbox by 10AM. I shoot for inbox zero and get there occasionally. But more importantly there isn’t the nagging feeling of hundreds of unknown, unread emails lurking on my machine. Again, the email isn’t exerting control over me, I’m in control over it and its satisfying.
Photo credit from www.vectorportal.com
Three things I learned this week
While I wait for my daughter to fall asleep so I can go for another ocean side run in Gloucester today, I thought I’d jot down three things I learned this week at work at HubSpot. It was a busy week, with lots of new faces as the Performable team made itself at home in our offices.
1. No sacred cows. We’ve got two products now at HubSpot. They both have their strengths. The thing I’ve kept on reminding myself, as we begin to understand the two products and how we can bring them together is that we shouldn’t hold onto any sacred cows as we work to create the best product for our customers. We shouldn’t care if it came from HubSpot or Performable or neither. We can’t be overly attached to our own handiwork if its not going to be the best option moving forward. Its easy to get defensive in these situations.
2. Don’t forget that new people don’t know everything about you. I did my first customer webinar on Friday. I got a compliment or two on my ability to give a demo. It was like they were surprised I could do it. And it reminded me “Hey, most of these HubSpotter only know me as a product builder”. For 4+ years at IBM I was in product marketing doing things like creating messaging and scoping out demo scripts and getting on planes to give talks at trade shows and gatherings of customers. These are all things I don’t typically do at HubSpot. So don’t forget, no one knows your other skills if you aren’t a little pushy about using them (assuming of course you want to use them!)
3. Patience. Lastly, I was reminded of the merit of patience. Good things happen sometimes simply by putting your head down and working hard.
3 Tidbits I learned from building Benchmarks at HubSpot
At HubSpot, the team I worked on recently built a “Benchmarks” application inside our product. This of course entailed calculating lots of benchmarks based on our customer data.
I wrote up three little learnings I found on the HubSpot blog.
Why I loved “Born to Run” . . . and Didn’t
I recently read Born to Run by Christopher McDougall. Its spent a good amount of time at the top of best seller lists so I had some high expectations for it. It was a decidedly mixed read though: I absolutely loved some parts of it and others left me disappointed.
The good
The book came across as 5 or so major main discussions, stitched together I really enjoyed two of the sections: the profile of the Tarahumera people of Mexico and the discourse on human evolutionary biology as it relates to running. Both helped me understand more about myself, why I like to run, how I can find even more enjoyment from running and how to run better. I wished the whole book was focused on those topics. The National Geographic subscriber in me loved reading about things like the unique breathing mechanism humans have relative to other mammals and how it helps us run long distances.
Unfortunately for me, the other elements of the narrative weren’t as valuable.
The bad
The rest of the book was focused on ultramarathoning, profiling some specific figures in the ultamarathonong subculture, and telling the story of a race put together in the home region of the Tarahumera. Frankly, I found these threads of the story less interesting and not compelling. Maybe it was just me, but I just didn’t find myself empathizing with the chaacters or caught up on the excitement of the race at the end.
Net, I’m happy I read the book. It helped me understand myself a little better and that’s really valuable. 150 pages of extraneous uninteresting text is a small price.
Why I Run
I try to run at least 10 miles a week. I’ve been thinking a lot about why I actually do this, likely inspired by reading Born to Run.
When I run, I get time to think about the problems I’m trying to solve. But that’s not why I run.
When I run, I have a chance to work off my frustration. But that’s not why I run.
When I run, I’m getting healthier. But that’s not why I run.
When I run, I sleep more soundly. But that’s not why I run.
When I run, I get the satisfaction of not paying for a gym membership. But that’s not why I run.
When I run, I have fun talking with my running mates. But that’s not why I run.
When I run, I love getting the fresh air and sunshine. But that’s not why I run.
When I run, I discover new places. But that’s not why I run.
When I run, I’m more patient and enthusiastic with my kids. And that’s the best effect. With my super-wife traveling more and more these days, I’m home alone with two pre-schoolers every morning and every night. The days I run, I’m drained physically, but mentally I find I have much more patience with the 23 requests to go to the bathroom after being tucked in, with the 501st episode of “he hit me, ow”, with the 2 millionth time I’ve been posed the question “How come?”
When I run, I’m a better dad. Corny, but its what gets me out there every day.
You?
Recursive Images
And now, for something completely different . . .
Every day at HubSpot, I have a ‘standup’ meeting with my scrum team. We have a team member in Egypt so we typically do it over Skype, video and all. Nothing amuses me more than projecting my colleague in Egypt’s image on the screen and pointing the camera at the screen so that the Skype’s view-finder window shows our colleague recursively, i.e. a picture of my colleague shows a picture of my colleague which shows a picture of my colleague which . . . . I don’t know why, but I find this hilarious.
I was reminded of this today when I read xkcd.com today:
Which reminded me of my favorite post from deadspin.com (before I stopped reading it) and the image of Juan Rivera from the LA Angels looking at himself recursively:
I love the Rivera’s body language there.
Some simple googling led me to learn that this effect is commonly known as the Droste Effect, named after an advertisement from 1903 that employed it:

My earliest recollection of my fascination with this phenomena was with a cookie box from McDonald’s. It showed a picture of Ronald and friends, with Ronald holding a box of cookies which showed a picture of Ronald and friends, with Ronald holding a box of cookies . . . my 7 year old brain was totally fascinated. Unfortunately I couldn’t find an image on the interwebs (shocking, I know!).
Anyone else have a favorite example of this (non MC Escher division)?
Edit: From reddit today: 


