Josh Payne's Unstructured Thoughts

Last weekend, I was worried about the operational state of our analytics pipeline at work. For the uninitiated, its basically a series of ‘jobs’ that process records from each page view on our customers’ websites and tabulates all sorts of reports from that data. It runs 24 hours a day. It usually runs smoothly but occasionally it’ll have a problem in production that we need to solve. And since I’m the earliest riser on the team that is responsible for it, when I wake up, I check on its state. Lets level set though — I’m a product manager, not an engineer. I’m not doing anything heroic here. Pretty much the best thing I can do at 6AM when I wake up and see that the processing is heading in the wrong direction, is say “crap, someone smarter than me needs to wake up to fix this”.

So part of my responsibility is monitoring the well-being of our operations. But another part of my responsibility at 6AM is my kids. I can happily live on less than 7 hours of sleep which means I’m ready to rock and roll with the them on the weekends when the kids are up at 6 AM. So last weekend, I find myself downstairs with my 5 year old. I’m moderately concerned with the state of the pipeline in our production environment. I have my laptop out and am checking out our monitoring tools. But my 5 year old son is, you know, 5. And he’s hungry because its been something like 12 hours since the last time he ate. And expecting pancakes.

So I do what every geek dad would do. I give a 30 second explanation of what a flow diagram is and the meaning of the colors on the graph of the flow diagram in question. And then I put him to work. Because there are pancakes to be made. The following ensues:

I don’t generally post things publicly with my kids, but I thought this was cool (and I figure he’s facing away from the camera).

Every production process should be this easy to monitor and watch. Of course my 5 year old (or me) can’t do anything if those boxes turn the wrong color, but  . . . my 5 year old understands what’s happening and is pretty close to being able to identify if a Bad Thing is happening. The data visualization is simple and it works. Its intuitive. The design is pretty rudimentary in terms of styling but the visualization of the data delivers a clear, easily understood story. Not too much to juggle and a mere mortal gets it quickly. It solves a key problem – informing a broad set of mortals in our organization how key operations are executing. The more people in your organization who understand the state of your most complex operations, the better you can all do your jobs. I love love love it when I stroll into HubSpot’s support area and see folks looking at this and other operations graphs. Your entire operation works more efficiently. Support teams can respond in real time to customer inquiries without “asking the engineers” and engineers are focused on either moving your product forward or solving the devOps problem du jour without major distraction.

Anyway, thought this was too good to not share. Enjoy.