Tess Flynn: Drupaldelphia 2019 Recap
TEN7’s DevOps Tess Flynn aka @socketwench gives us her observations of the Drupaldelphia 2019 conference she recently attended, as well as a summary of her helpful session, “Return of the Clustering: Kubernetes for Drupal.”
Host: Ivan Stegic
Guest: Tess Flynn, TEN7 DevOps
In this podcast we'll discuss:
- What it takes to get a good Jedi costume
- Tess’s Trilogy of Talks
- Summary of Tess’s Kubernetes session
- How camps are not just about Drupal and not just about developers anymore
- Short camps = no keynote
- Help desks at the conference - what a great idea!
- Avoid Deep Hurting! Deployments beyond Git (Episode 2 of the Tess Trilogy)
- Return of the Clustering, Kubernetes for Drupal: slide deck
- Return of the Clustering, Kubernetes for Drupal: recording of the actual session at Drupaldelphia
- Docker Swarm
- Get $50 credit towards a new Digital Ocean account
IVAN STEGIC: Hey everyone you're listening to the TEN7 Podcast, where we get together every fortnight and sometimes more often, to talk about technology, business and the humans in it. I'm your host Ivan Stegic. Let’s talk about the Drupaldelphia 2019 conference that happened recently in Philadelphia. And, joining me to give her thoughts on the camp is Tess Flynn, a seasoned guest here on the TEN7 Podcast. @socketwench welcome back to the podcast.
TESS FLYNN: Hello.
IVAN: I’m going to have to make you the guest host when I’m out of town. [laughing]
TESS: [laughing] Sure, why not. We could do one on terrible movies. [laughing]
IVAN: [laughing] That would be actually awesome. We should totally do that. All right, Drupaldelphia. What’s in a name? So, this is the annual Drupal Camp in Philadelphia, Pennsylvania, and this year it was part of something called Philly Tech Week 2019. And you Tess, were there representing TEN7. What do you know about the relationship between Philly Tech Week and Drupaldelphia?
TESS: Not much, actually. I was so focused on other things during the week, I didn’t have any time to really investigate it deeply. But I noticed that some of the other attendees didn’t know that either, [laughing] so at least I was in good company.
IVAN: Okay. And were there any signs around that referred to Philly Tech Week, or was it kind of just insular?
TESS: No, not that I noticed. I think that it was just part of the longer weeklong event. Like, some organizations will have this kind of co-boosting agreement, where we’ll talk about your event and then you’ll talk about our event, and it’s good for both of us, and we don’t have any other commonality. And there’s nothing wrong with that. That’s good.
IVAN: That is good. Have you been to Drupaldelphia before?
TESS: I have not. This is my first time. It just showed up across my board, and I went, Huh, I haven’t done that one before, sure, why not?
IVAN: And we have a great talk that you have produced, and so, as a company, we’re making sure you get out and tell that to as many people as possible. So, it was a good opportunity, I think.
TESS: It’s the first talk that I’ve been giving in costume, which is hilarious.
IVAN: Right, I heard about this, and you talk about it in the session, in the recording. So, tell us about the costume.
TESS: So, the thing is that, when you’re doing air travel, it’s always best to only have carry-on luggage, because you never know if you’re going to have a delay, or if your flight is going to get lost, or if your luggage is going to get lost, or something. So, you should restrict yourself to a roll-aboard and a backpack. Now, I’m an old hat toward business travel, so, I’m used to the standard, roll-aboard-backpack modality, but it does limit what you could bring with you when you’re traveling. So, the main concern for coming up with an outfit is: how can I do this so that I minimize the amount of potential luggage space that I take up, while still being evocative of the thing that I’m trying to riff on. And after thinking about this for a while, I actually did come up with something that was rather clever. The easiest thing to do was first find a plastic lightsaber. Not an electronic lightsaber, not one that lights up, not anything else.
The recommendation is that it has to be a lightsaber that’s made of plastic, that fully collapses into the hilt, and has no other electronics. Why is this important? Because air travel. Because we don’t want to have to pull the lightsaber out and put it on the scan deck for everyone else to look at. When it’s just plastic, it’s just plastic, no one cares about it, just put it in the bag and no one cares. Fortunately for me, I found one that was The Last Jedi branded, but it’s actually a model of Luke Skywalker's lightsaber, that was fully plastic, no electronics, completely collapses into the hilt, on Amazon for 10 bucks.
TESS: And that was really as far as I was going to take it, because having a prop in the talk isn’t necessarily the worst thing. But then I started getting a little bit ahead of myself and I go, Well, what else can I do? And it occurred to me that one thing that I could do was, I should probably do up the outfit a little bit more. And, one of the things I decided I could do to do up the outfit a little bit more is, what is one thing that a lot of Jedi have going for them? And so, I thought back to all of those hours that I spent watching all of those movies, and all of the comics I read, and everything else, and go, They have this thing for cloaks. That’s it! They have this thing for cloaks and robe-like things. Well, I can’t really do a robe-like thing, because that would require a lot more layers in order to do everything. However, the problem is that, in order to really be evocative, I still need the exterior robe. So, I went again to Amazon, and I found a $20 brown cloak with a hood that looked pretty darn close to what a Jedi would normally wear [laughing], and that’s all I got.
And, so, I am out on stage wearing my normal talk outfit, the infamous skull dress, and wearing this huge brown cloak and carrying a plastic lightsaber. And, I actually stand in front of the room I’m giving the talk in before it starts, ushering people in, acting like a complete fool. Like, everyone going like, What am I getting into here? Why is this person in costume? And I’ll show up to the camp in costume for the entire camp [laughing] because it’s fun and it’s different and it’s weird, and one thing that is really kind of useful is, this is meant to be a fun thing. It’s meant to add more flavor to the talk, and the talk is full of my style of corny jokes. So there’s nothing wrong with me also amping up the corny the entire time. This whole thing about me amping up the corny, actually started in, I think 2015, in DrupalCon where I showed up with a giant inflatable whale, and I literally carried that around with me until the talk started and then I gave it away within the first few minutes. [laughing]
IVAN: I love how involved you get in your talks and how personal your slides are, and the comics that are hand drawn and the characters that you have, they just really make it a joy to listen to you speak. And, I wondered, because I haven’t actually seen you give this talk yet, I will absolutely be going to it at TC Drupal Camp when you give it, and in my mind I wasn’t sure if it was a black cloak or a brown cloak, but I was pretty sure you went with brown, so I’m glad to hear you went with brown. I guess the question I have is, did you have a hoodie?
TESS: I didn’t have a hoodie, but the cloak has a hood on it. So, when I’m outside the room ushering people in, I will put the hood up and start waving the lightsaber around, and usually that’s enough to let people get the idea of what I’m doing. [laughing]
IVAN: That’s really great. Ok, so let’s actually stay on this track. We’ll get to the details of Drupaldelphia in a sec here, but since we’ve already started talking about your session, let’s keep going. So, your session is called "Return of the Clustering, Kubernetes for Drupal", and it’s part of a trilogy, and I’m going to say I think first of all it’s amazing that it’s part of a trilogy. So I’d like you to kind of just tell us what Episode 1 and Episode 2 are about, and my very technical question thereafter is going to be a follow-up, because this is a trilogy of trilogies, but we can get to that in a second. Let’s set up the Episode 1, Episode 2 part of this.
TESS: So the whole, "it’s a part of a trilogy" thing was actually a bit of a joke on myself. One of the first talks that I started doing outside of Minneapolis was on "Ride the Whale: Docker for Drupalists", and the idea was to teach people how to build your own Docker-based local development environment, and how Docker in general worked. And, that was the first start of it. Then the next talk that I gave was “Avoid Deep Hurting! Deployments beyond Git,” which introduced how to build your own continuous integration system using just open source, free stuff. Free stuff like Ansible, free stuff like Gitlab CI, and the idea behind that is now you have these two different pieces that don’t seem related, but become related in this talk.
I kind of sat back and thought about all the talks I’d given over the last few years and realized, Oh, geez, I made a trilogy. Oh, God, what did I do? When that occurred to me, I also knew what the whole theme of the talk was going to be, and because it was a trilogy, I decided, Okay, fine we’ll call it "Return of the Clustering" and we’ll do a whole Return of the Jedi motif, because why not? [laughing]
IVAN: Exactly. So, we’ve set up the trilogy, now "Return of the Clustering" is a wonderful evolution, in my opinion, from your very beginning of trying to figure out how the heck are we going to set up Drupal in an easy Docker container or set of containers locally, and basically fast forward to the real meat of what we’re trying to do, and that’s run Docker in production? Right? So, give us a high-level description of "Return of the Clustering."
TESS: So, we first start out with describing, Wouldn’t it be great if we ran Docker in production? And we dissect the various problems with that. There are security problems that come inherent with running a Docker-based workload that you have to be aware of, and most people who just use Docker out of the box might not have ever considered these facts. And, it’s different than traditional server management, because there are a few different additional factors that come into play with how Docker executes things. Then you also have to worry about how you’re going to actually get the workload onto the cluster as well, and how you’re going to orchestrate it. One of the biggest problems is, it’s really easy to stand up Docker on a single server and have that single server run an entire workload of multiple containers.
There’s nothing wrong with that. You install Docker, you install Compose, you write a Compose file, you stand it up, you do some security things to make sure everything is loaded correctly, and there you go, you’re done. But, the problem is that your scaling is only vertical. You can only make your server bigger. You really want to make your scaling horizontal and add more servers. And this makes things very complex, because if you have to coordinate multiple Docker hosts together, doing that manually is not fun, and it’s not DevOps either. It’s not automatable in a very easy way. Fortunately, there are container orchestrators out there that know how to do this for you. So we talk about Docker Swarm and what the advantages and disadvantages of that are, and then we introduce Kubernetes. We talk about how the Kubernetes model is different, then we architect out an architecture that will run a Drupal site in Kubernetes. And, this is actually a subtle point that I keep having to remind myself and others, is that the Kubernetes model is so complicated, with so many little details, and so many different things in it, that you really get lost very quickly. And it took me a year to figure out which bits of the Kubernetes’ model work for Drupal, and which bits we don’t need to talk about. Once I understood those parts, it was easy to build an architecture from those minimal amounts of pieces.
So, we review what those pieces are, and we describe them. Then we talk about how to technically implement them within Kubernetes, and why Kubernetes is kind of nifty through its use of YAML. So, what we’re going to do is, we build all of that out. But now we have another problem, which is, if we want to build the dynamic cluster which supports multiple clients, multiple sites, that’s a lot of YAML we have to manage and now we’ve just introduced a problem that I covered in Avoid Deep Hurting, which is, we have introduced humans back into the mix. We need to take the people out of the mix and let the technology handle it for us, because it doesn’t get tired, it doesn’t make typos, it doesn’t need three more coffees in order to get through the day—at least not most days. [laughing] So, we combine Ansible and Kubernetes and Docker to build out an entire cluster in an automated fashion, by just changing a few different variables, and we run that on Digital Ocean.
Then we also found out that the problem is, to really effectively leverage Kubernetes for Drupal, you can’t just put an Apache container out on Kubernetes and then throw the site on it, and then update it in place like you did with traditional server management. It’s not really the way that Kubernetes wants to do things. So, instead, we end up having to build a custom container that contains the site code already, and then run that on Kubernetes directly. And that’s a much more cloud-native workflow, and it’s a bit of a paradigm shift from what a lot of people are used to. But, each one of these pieces works well together to create an effective, open source, minimal Kubernetes production cluster for Drupal.
IVAN: I love that it’s minimal. I mean, you just put what you absolutely need out there. And that’s kind of the philosophy and it reduces your security work, quite honestly, and it also reduces the attack vector. So, I’m so glad to have heard this summary about the talk. I hope that people go out there and watch the recording, and if you have an opportunity to attend TC Drupal Camp or Flyover Camp in Kansas, you’ll be at both of those places giving the talk again.
We went through your talk, let’s talk a little bit more about Drupaldelphia 2019. So, it’s a one-day camp, and it kind of looks like there were six tracks, which is a large number of tracks for a camp.
TESS: It was a surprising amount of things you could do in every time slot.
IVAN: And it feels like the tracks weren’t just Drupal, right? I mean we talked about this with Chris and Dan [organizers of TC Drupal Camp]. It kind of feels like there is this conscious effort in the camps, at least the ones I’ve been paying attention to recently, to be not just about developers, and not just about Drupal. So, what do you think about that?
TESS: I think that’s really a good strategy and it’s a lot more holistic than we’re used to going forward. I think that our industry is still getting used to the idea that there is an internet that we can connect with each other and research things, even though we do it every day. I don’t think that the cultural and emotional impact of that has really entirely sunk in. And a lot of events are changing to reflect that attitude, that it’s not just one piece of technology that we need to deeply investigate anymore. It’s now: how does this one piece interact with a whole bunch of other pieces? And some of those pieces aren’t technology. A lot of those pieces are people.
IVAN: Yeah, the people aspect is so important as well, and I’m so glad to see those tracks are appearing on the local camps. And, so, one day, six tracks, five sessions in the days, so about 30 sessions total in the whole camp, which sounds about exactly the same as what we’re doing in TC Drupal this year on the one day of the camp. I didn’t see a keynote on the schedule, an official keynote. I didn’t see intro and outro, welcoming remarks and ending remarks. Is that right? There was no keynote?
TESS: There was no keynote. There was a brief intro, and then the session started. And I think that actually worked fairly well for this camp, because I think that the shorter the camps, a keynote is less effective. And, this is a bit of an attitude change, because I remember when TC Drupal first didn’t have a keynote, and I kind of missed it. I kind of wanted that back, because one thing that was kind of nifty about having a keynote is that, it’s there to introduce you to the talk, and get you excited, and get you ready for the day. But for a one day camp, it’s not really that necessary, is it?
TESS: You’re already there. You already are invested to do the day, and there doesn’t seem to be much of a need to really do that. It might be that keynotes are a thing that are best suited for multi-day events.
IVAN: Yeah, that’s kind of what I’m thinking as well, multi-day events that maybe need higher attendance, and so you almost use the keynote to attract attendees that you hope stay on. So, maybe there’s a celebrity, or someone who’s done an awful lot of contribution that has the keynote. And I’ve always thought of that as an attractor, but it almost eats into the day if it’s a one-day camp.
TESS: Yeah. It usually takes a good hour and a half, and you could get a whole entire slot and a break in otherwise. And from an organizer perspective, attracting a keynoter is often very difficult, because if you’re a keynoter you usually should compensate them in some way, even if it’s just paying for lodging and a flight. But that can be a bit of an impediment, especially if you’re a smaller event, or a less established event, you might not be able to get someone to do a keynote. And, I think it might just be not as necessary anymore. It might just be better to have everything else.
What was also kind of nifty about Drupaldelphia is that they had these help desks, as well. Every time slot had an accessibility lab, a Drupal and Composer help desk, and a contribution workshop room. And I thought that was spectacular, because it allows people to drop out of the sessions and work on other things or get directed help. And a Drupal and Composer help desk, I thought that was a brilliant idea, because wow Composer. Composer can really, really ruin your day. [laughing]
IVAN: Yeah, I agree. So, is that how BoFs and Contributions will handle theirs basically during the whole day and maybe as options and alternatives to the sessions that were already scheduled? Is that how it was handled?
TESS: No, there was also a distinct BoF room, but it did share with that particular room, which I think might not have been necessarily the best choice. I think that it would’ve been a bit better to have a dedicated BoF room, and then a dedicated help desk lab room, because that would allow people to be less interrupted by people who are doing a BoF. And sometimes BoFs end up being sessions that were submitted, but weren’t accepted for whatever reason, and those could be kind of distracting.
IVAN: And animated. Sometimes you get people who show up in cloaks and sabers in those BoF sessions too. [laughing]
TESS: Only if it fits in my carry on. [laughing]
IVAN: [laughing] It sounds like you had a good time there, Tess. That’s great. So, it was on a Friday at the end of the work week but in the middle of the day. What was attendance like?
TESS: There were about 250 people who attended. It was surprisingly packed. I think this is one thing that I noticed that a lot of these one-day events have kind of a side effect that they can concentrate their attendance. A multi-day event actually has kind of the reverse effect. It spreads out the attendance to multiple days and makes it more difficult to actually advocate it. Particularly if one of those days is a weekend. If one of those days is a weekend, a lot of developers, depending on the age
bracket of the segment that you’re going for, and Drupal is tending to get a little bit more middle-aged than it was like 10 years ago.
IVAN: Oh, come on. Come on. [laughing]
TESS: Hey, I’m being honest here. I’ve been to a lot of events and I’ve looked at the age of the people showing up and it’s like, Yeah, I’m seeing it.
IVAN: I know what you’re saying. People are little wiser, a little grayer, a little more family, families are attending. Yeah, it’s true.
TESS: And, as a result, having time on weekends is kind of a difficult call, even a difficult ethical call, because some people will do this for work. And now you’re asking them to do work on a Saturday. That could be a little bit of a difficult call, whereas if it’s a one-day event that’s during the week, then it’s fairly concentrated. And at that point you know that it’s just a workday that’s being used, and if this is aligned with your career, there’s nothing wrong with that. And, then it doesn’t interfere with your weekend and that critical amount of recovery time, that we all need at the end of a work week.
IVAN: I agree. I think it’s good for mental health to do it the way that it was done. What was the location of the camp? What was that event space like? Where was it located?
TESS: I think it was called The Hussian School of Art; I think. Originally when I found the building, I went to the wrong door and some very nice security people at the door were like, “Where are you going and what the hell is Drupal?” [laughing] So I had to give them my spiel really quickly and then they said, “Oh, oh, you want that? It’s probably around the corner. So, go down to the end of the block and take a left.” It was really helpful. And then later on when I went to get some coffee I passed by there and I found a door. I found a sign there that said that you should always go around the corner. So, no one else had to repeat what I did in the morning. [laughing]
IVAN: Okay, good. That’s called iterating, I think. Were you wearing your cloak at the time that the security guards were giving you directions?
TESS: I think that I stuffed it in my bag while I was taking the cab to the event, because the only hotel I could get was actually right next to the airport. [laughing]
IVAN: Oh, my goodness.
TESS: And so, I had to take a cab ride to go into town and I didn’t want to seem weirder than I usually am to people. So, I stuffed the whole thing in my backpack and fortunately a plastic lightsaber and a cloak does not require that much space intentionally. We covered that already. So, it was kind of handy.
IVAN: So, a successful space then you felt. Was there kind of an atrium or a gathering space for everyone where there was lunch? Or, how did that work?
TESS: There was this semi-open area. We had the quick intro in that semi-open area. And then they pulled out some moveable walls to enclose that to make another room, which was nifty. And there was a nice little gathering spot where you could see the tables, and you could get lunch, and you could interact with different vendors that were at the event. That was all kind of nice, and then there was a couple rooms down some hallways, and yeah, it was a nice, cozy space and I rather liked it.
IVAN: So, overall impression of the event then, seemingly positive?
TESS: Yeah, I really enjoyed it. I thought that was a great event, and I think that more people should go to it.
IVAN: So, definitely a candidate for going to next year. We’ll have to wait and see what the next episode looks like of the series, I guess. And whether there will be a next episode?
TESS: We’ll find out. There might be a case study afterwards where we talk about what we actually did that might be a little bit more interesting to people than an intro to Kubernetes itself, but I think that it’s necessary.
IVAN: You know, that’s a great idea, to do a case study. I know we’re working through actually implementing something live right now, so be taking notes Tess, because it sounds like you already have an idea for the case study talk.
TESS: (laughing) All right.
IVAN: All right, all right. Well, thank you so much for spending your time with me today, again. It’s truly been my pleasure to have you on. Tess Flynn is the DevOps Engineer here at TEN7 and she was at Drupaldelphia 2019, giving her talk, "Return of the Clustering, Kubernetes for Drupal." The slides are online and a recording of the session itself is available as well. Just visit this episode's webpage for the links. The URL to this episode is t7.io/ep60.
And, don’t forget, if you’d like a credit towards your new Digital Ocean account, just go to ten7.com/digitalocean and follow the link on the page to redeem it.
You’ve been listening to the TEN7 Podcast. Find us online at ten7.com/podcast. And if you have a second, do send us a message, we love hearing from you. Our email address is email@example.com. Until next time, this is Ivan Stegic. Thank you for listening.