by Katie McLaughlin

“So, Katie, what do you do?”

K: “I work in tech”

“Oh, ok, so you make (apps|websites|games)?”

For any given dinner party, if you say that you work in tech, many people expect that you are doing something ‘cool’; making computer games, websites, the next iPhone app, or the like.

Not all of us get to work on the next latest and greatest thing, however.

K: “Uh, no, I keep the cloud running.”

Thankfully, most people at a sufficiently diverse dinner party know of this ‘cloud’ concept and leave it at that, but occasionally, there’ll be that one person.

“Oh, you’re devops! What’s your stack?”

This is the question I hate to answer. It normally goes one of two ways:

K: “It depends, I’m a polyglot, so I just fix what’s broken.”

“Oh, ok”.

…

K: “It’s legacy, so there’s a bit of Perl, PHP..”

“Ew, you have to use PHP!”

Many people in tech aren’t working with the latest and greatest tools, making the next big thing. A lot of us work with legacy systems: systems that have stood the test of time, where the cost to replace them highly outweighs the cost of keeping them up. Especially in infrastructure deployments, there’s an extremely high cost and risk in changing things, so people are paid to learn the previous best thing and keep it stable.

I’ve spent many hours spelunking through Perl applications that serve as ticketing systems; PHP systems that run behind the monitoring systems; bespoke applications grown internally that serve needed purposes, where no off-the-shelf or SaaS could compare in feature parity.

It’s extremely satisfying to get to the bottom of a particularly annoying bug. In one instance, I spent a whole lot of time diving through Perl in order to find the source of a particular issue: we could link tickets in our incident tracking system, but any time we did, a link to a mythical ‘Ticket #1’ would also be added. This made for interesting effects, as the linked tickets liked to also list their parent and child ticket links. It created a very boggy system with slow page load times.

I dove through the local fork of the code base, comparing it to the open source upstream code, now several years past what was installed locally. I found the bug had been fixed in later versions, but the code base had shifted, so it wasn’t easily patchable. I also found a whole lot of internal customisations in our environment that made the software work for internal business cases, but prohibited upgrading. I eventually found the issue was related to how there’s no such thing as ‘booleans’ per se in Perl. The 1 value was being interpreted as literal integer rather than True, and it was using that as a primary key for the ticket, which was how the ‘Ticket #1’ linking situation happened. Instead of the logic asking “Is there a link? True!”, it was saying “Is there a link? 1!. Ticket #1? Ok then! link”… A bit of dynamic backporting later, and I’d fixed the issue, much to the merriment of my colleagues. They were glad to see the performance improvements, and thanked me for my ‘sacrifice’ in having to use Perl.

It’s not just internal bespoke versions of applications that can lead you to learn languages you don’t otherwise ‘need’ to know. There are so many legacy tools in the standard Linux operating system that to work in the world of DevOps it really helps to have a general understanding of 15-25 year old systems.

For example: man (manual) pages. Yes, you could search Google for the HTML-rendered versions of these things, or you could use the man command on the system. The man page for any particular app has it’s own specific way of working: man serves up the file on disk using a program called less, which allows such things as pagination and searching by keyboard controls. less itself is a rewrite of an older file pagination tool, called more (because less is more [ba-doom-ching!]). Plus there’s the arcane art of the man-page section number, because there’s a difference between, say, the system call stat and the shell script stat, sometimes referred to as stat(1) and stat(2), which are two separate entities, each with their own man page.

Or Git. A lot of companies nowadays are working with GitHub, either in private or in the open, but the underlying technology in Git is over 10 years old. And most of the functions on the command line are just shell scripts! The git submodule function is a 1000+ line shell script that calls out to other Git functions, some of which are more shell, and some of which are compiled C. Until recently, submodule contained embedded Perl code to handle some of the more interesting regular expressions required. Which means that you can’t use git submodule unless you have Perl installed on your operating system. Normally this wouldn’t be a problem unless you’re using a containerization operating system such as CoreOS, which by default doesn’t come with any scripting language environments. The shell script Perl dependency was removed around about git version 2.7.0, so depending on which version ships in your operating system, you may or may not have to have Perl to use the complete git functionality.

The only reason I know the specifics about the way that man works is because I read it in a book. But I learnt about some of the internals of git and its previous reliance on perl by hours of debugging and code spelunking. The latter was much more fun.

Coming back to the dev side of DevOps; because I’m not using the latest bespoke Node.js functional programming widget that’s all the rage, I’m treated as inferior. Even worse, if I *was *using Node or other JavaScript systems, even these are seen as beneath other programming languages because of the legacy that ‘JavaScript’ has. The fact that I’ve experienced dinner-party conversations from people with only passing knowledge in technology, and not working in the industry, who “know” how bad JavaScript and PHP can be is somewhat concerning. The fact that the “horribleness” of these languages transcends our sphere and into common lay knowledge is not something I expected, and it worries me.

There are people who have to keep the lights on, keep the systems running, and that’s what I’m good at. To be taken as subpar because of the technology I use is degrading.

Aurynn Shaw discusses this in her article “Contempt Culture”. She describes how the act of degrading a language is to degrade the people using it. She also outlines the concept of ‘language bashing’ or ‘platform bashing’ where it’s so unquestioned to talk down other technologies in favour of your chosen one, because some of them (especially PHP and JavaScript) are seen as easy marks.

I admit, I’m not immune to this kind of behaviour. Before Perl I worked in the Oracle and Windows space, two words that are marked as ‘evil’ in some open source circles, and two words that seem to keep being picked up by auto-recruiter bots for years afterwards. As a coping mechanism, I’ve used self-deprecating language in the past. Now that I’m no longer working in that space, I’m can pass as having ‘done my time’, it can be far too permissible to take cheap shots at those still working with the technology that I’ve long since left behind.

Many people have no choice in their tech stack. They have no option to re-write. They have to keep the machines running and the wheels turning. They gain experience in these stacks, and then actively choose to continue working in these technologies, even across companies. To tell them that because their tech isn’t ‘cool’ or ‘hip’ is erasing their experience as irrelevant.

As much we can see our own platforms as perfect or without fault, some tools are objectively malformed or broken.

I give a talk which has a section dedicated to showing ‘wats’ in no less than a dozen languages, using the concept of a ‘wat’ by the talk of the same name by Gary Bernhardt. Some of these are a kind of polyglot blues—Perl has specific numeric equality operands that silently coerce values, Haskell has a length function that outputs unlikely things for tuples—but some are more clearly wrong. In Powershell, the “greater than” operand has been implemented, but the “less than” operand has not, because < is also used for input redirection. The order of precedence in Ruby means that and and && aren’t processed equally—you’ll have to know which order will do what you wanted. Not to mention the entire talks I’ve seen dedicated to ‘JavaScript wats’, including Bernhardt’s. And if there isn’t already a talk entitled “12 ways to allow PHP remote code execution in 10 characters or less”, I’d be highly surprised.

But there is a major difference between attacking a language’s faults and attacking a person for having chosen, or not chosen, a language. Programming languages are hard to create, and the design decisions behind creating languages are just as difficult to get right. Some languages, like PHP and Python, have the ability to increment major versions and uproot a lot of functionality and allow for major reworking. Some languages, like JavaScript, really can’t take that approach. Its ubiquity and overlapping deployment on the web means that design decisions made in the inception of JavaScript back in 1995 still carry over to this day.

We need to break our pack mentality that different segments of tech are better than others. At the end of the day, we are all trying to solve problems and provide solutions to the best of our ability.

Most at the dinner party I started with can’t tell the difference between when they are using the web or various apps, or which underlying tools built the technologies they use, so why must we create artificial barriers that impede progress? Why must we have these walls?

As a polyglot, I interact with many different segments of the tech community, across languages and up and down the stack. I also have experience with legacy tech, so I can work on older and newer stacks, across the expanses of time (whee, I’m a time-lady!). There are so many ‘solved’ problems that keep being ‘resolved’. Nothing in tech is ever new. The legacy systems can teach the new-age systems as lot, if only they are willing to learn. The reverse is true, too. The new-age systems can show their worth, and some of their lessons can be retroactively applied to legacy systems.

If we can share our knowledge, and not degrade each other for their choice of dialect, then we can create a more advanced knowledge base to draw solutions from, and improve all our systems for the better. At the end of the day, someone needs to keep the lights on and keep the systems running. When you have a nicely-tuned and humming system, it’s awesome. It’s what I’m good at, and it’s gratifying as hell.

Katie has worn many different hats over the years. She has been a software developer for many languages, systems administrator for multiple operating systems, and speaker on many different topics. When she’s not changing the world, she enjoys cooking, making tapestries, and yelling at JavaScript and its attempt at global variables.

Illustration by Victoria Wang.

We have a print edition too! Find this issue in the shop.