Over time I’ve compiled a small library of scripts for various p4 functions that have been written time and again at different sites…mergetool is one of them. This little tool accepts a merge target (”yours” in p4-speak) and projectile (”theirs” in p4), labels both, performs an integrate, and performs a “safe” resolve -as. It logs any failures for you to resolve by hand, or submits the change set if the resolve completes successfully. It does this with a bunch of logging in a well-organized, date-stamped directory suitable for archiving (or splunking).

Surprisingly, this is not an uncomfortable place to be. In 11 years in industry I’ve worked in a variety of organizations: the now-bankrupt dot-com best known for putting an ad with a naked guy up during the Super Bowl, 2 major marquee names with vastly differing corporate cultures, a security start-up stocked with emancipated-minor hackers. Aside from that doomed dot-com — which had a surprisingly strong gender balance throughout technical roles and a culture blessedly free of gender-based intimidation at all levels — Splunk may be the most comfortable place I’ve ever worked. There’s no creepy tokenism (unlike stories I’ve heard about certain other bay area employers), That Guy Who’s Never Seen A Girl Before doesn’t work here…and as far as I can tell, no one really gets harassed except Amrit.

Perhaps a better testament for the dev culture than my opinion — because, frankly, I’m pretty weird to start with — is that other women in the company seem to be pretty comfortable visiting the dev area, either on work errands or just to take a break from the sales-focused environment upstairs. Frankly I can’t imagine that happens too often in the bay area…and more’s the pity.

By the time 2.1 was on its way, we’d decided to switch to native packages, and our list of platforms had expanded to include Solaris on Intel, with several more on the horizon. We also wanted to provide the “rail tarball” distribution we continue to support, in part so that QA could get started before the packaging automation was complete.

What is that packaging automation, you might ask? Obviously writing custom code to package each platform (not to mention spec or pkgmap files in each platform’s native format) was not a very maintainable solution. Instead we use a locally modified version of Easy Software’s EPM package manager. After a little work, EPM lets us use a common set of list files to create relocatable packages using common pre- and post-install scripts across all of the 9 platforms we now build on. We’re able to control every file and permission that goes into the packages, and in most cases we can add packaging for a new OS platform with a minimum of work (for something very different we haven’t previously had in house, like AIX, more time might need to be spent cleaning up EPM’s support for the platform). We’ve piggy-backed creation of the “rail tarball” distributions on the EPM list file structure, so those packages too are completely defined. EPM itself is built during the Splunk build process like any other 3rd party dependency, so any new patch to the tool are available to the build systems almost as soon as it’s checked in.

The downside to this is a little loss of flexibility for some platforms; trivial changes to the RPM spec file or FreeBSD ports originpath have usually required modifying EPM’s source. It’s not a bad trade-off, though.

EPM has recently been made open source; check it out at http://www.epm-home.org . If you’re interested in my patches, feel free to drop me a line, but in some cases superior patches have been contributed at epm-home.org.

Thanks to my acquisition-happy former employer, Symantec, I’ve seen a variety of startup approaches to release engineering. Most frequently it seems some senior developer has a bug up you-know-where about how the build system should work, and some poor junior developer or sysadmin type person dutifully does the drudge work (usually by hand). At other sites, some very diligent and detail-oriented person creates and executes a process with a great deal of record-keeping and attention to detail but often not a lot of automation. Consistency across different build platforms usually isn’t a strong point.

Here at Splunk, things are a bit different. I called myself the plumber in the title of this post because that’s how I see my job: I create and maintain the plumbing that produces consistent, reproducible Splunk builds across all of our platforms, with as much visibility as I can muster. I see my contribution more as enforcing process through tools — ideally, tools that enable process in a way that is more convenient for everyone than “doing it wrong” — rather than personally pushing all the buttons and scribbling in all the logbooks. And I’ve had the good fortune to come into a culture that encourages this approach.

Whew. That’s a mouthful for an introduction. In the near future I hope to write a bit more about how the plumbing works, and some neat tools I’ve found along the way. I’m sure y’all will be waiting with baited breath.