General

#StateOfAutomation - Simple Solutions for Complex Problems

Published Jun 28, 2021

Updated Jan 31, 2023

3 min read

This article was written over 18 months ago and may contain information that is out of date. Some content may be relevant but please refer to the relevant official documentation or available resources for the latest information.

🤔 What in the world is Automation?

😱 How can we utilize automation to improve our developer experience and productivity?

🤯 Is it possible to over-engineer automation?

If you or your team have ever asked these questions, come dive into some of the talking points discussed at Modern Web's State of Automation event.

In this article, I will review some of the major points brought up during this conversation, hosted by Tracy Lee (@ladyleet) and Matthew Pagan (@mastapegs) of This Dot Labs.

Let's meet our panelists, and hear about what their teams have been working on!

👩‍💻 Alison Dowdney (@alisondowdney)

Developer Advocate from Weaveworks

Weaveworks dropped an early release of Weave Gitops Core, a tool that’s backed by Flux, an open-source tool backed by CNCF (Cloud Native Computing Foundation).

🧑‍💻 Taylor Dolezal (@onlydole)

Senior Developer Advocate from HashiCorp

Terraform, an open-source code software tool that enables you to safely create, change, and improve infrastructure, just hit 1.0 earlier this month!
Waypoint went to 0.4, which includes mutable deployments and makes it easier to work with Kubernetes.

🧑‍💻 Zan Markan (@zmarkan)

Developer Advocate from CircleCI

CircleCI has acquired Vamp, a release orchestration tool.
They’ve also released new features like enhanced ARM support, as well as Dynamic Config, which allows users to use jobs and workflows to not only execute the work, but determine the work they want to run.

👩‍💻 Tracy Miranda (@tracymiranda)

Executive Director from Continuous Delivery Foundation

Continuous Delivery Foundation hosted cdCon, which supports the tech community by helping organizations focused on tackling the diversity and inclusion gap.

🔑 Key Takeaway Points

As we work through the pandemic, and see all the workplace and cultural changes happening, we see that people are really prioritizing automation to help with distributed work and burnout. 🔥
Concepts and platforms such as GitOps and Kubernetes Projects have helped us expand automation outside of the coding realm for tasks such as simplifying the interaction between communities.
Automation helps with the technical aspect of our workflows, but it might not necessarily solve human or process problems in an organization.
It is possible to over-engineer automation! Maintainability is so important to keep in mind to make sure we've created a sustainable workflow for our future teams.
And no, Jenkins is not dead. Far from it. 😉

🧠 Resource Recommendations

📚 Accelerate by Nicole Forsgren, PhD, Jez Humble, and Gene Kim
📚 Working in Public: The Making and Maintenance of Open Source Software by Nadia Eghbal
💻 Perspective API: Free API that uses machine learning to help mitigate toxicity and ensure healthy dialogue online.
💻 kind: Tool for running local Kubernetes clusters using Docker container “nodes”.

🙋‍♀️ Have any questions or points to share? Use #StateOfAutomation on Twitter and talk with us!

📺 View the replay here for fun stories, great tips, and faces to names! 🤝

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

Susan Ma
@sukisupremo

Performance Analysis with Chrome DevTools

When it comes to performance, developers often use Lighthouse, Perfbuddy, or similar performance analysis tools. But when the target site has protection against bots, getting information is not that simple. In this blog post, we are going to focus on where to look for signs of performance bottlenecks by using Chrome Devtools. Preparations Even when access to automated performance analysis tools is restricted, the Network, Performance, and Performance Insights tabs of Chrome Devtools can still be leveraged. To do that, some preparations can be made. When starting our analysis, I recommend opening the page we want to analyse in incognito mode. So we separate the analysis from our regular browser habits, cookies, and possible browser extensions. When we load the page for the first time, let's make sure we disable caching in the Network tab, so resources are always fetched when we reload. Some pages heavily rely on client-side storage mechanisms such as indexedDB, localStorage, and sessionStorage. Cookies can also interfere. Therefore, it's good to leverage the "Application" tab's Clear site data button to make sure lingering data won't interfere with your results. Some antivirus software with network traffic filtering can also interfere with your requests. They can block, slow down, or even intercept and modify certain network requests, which can greatly affect loading time and the accuracy of your results. If the site under analysis is safe, we recommend disabling network traffic filtering temporarily. We strongly suggest just looking at the page and reloading it a few times to get a feeling of its performance. A lot of things cannot be detected by human eyes, but you can look for flickers and content shifts on the page. These performance issues can be good entry points to your investigation. Common bottlenecks: resources Let's start at the Network tab, where we can identify if resources are not optimised. After we reload the page, we can use the filters on the network tab to focus on image resources. Then we can see the information of these requests, including the size of the images, the time it took to load each image, and any errors that occurred. The waterfall chart is also useful. This is where you can see the timing of each image resource loading. We should look for evidence that the image resources are served from a CDN, with proper compression. We can check the resources one by one, and see if they contain Content-Encoding: gzip or Content-Encoding: br headers. If these headers are missing, we found one bottleneck that can be fixed by serving images using gzip or brotli compression while serving them. Headers on resource requests can tell other signs of errors. It can also happen that images are served from a CDN, such as fastly, but if there are fastly-io-error headers on the resources, it can mean that something is misconfigured. We also need to check the dimensions of the images. If an image is larger than the space it's being displayed in, it may be unnecessarily slowing down the page. If we find such bottlenecks, we can resize the images to match the actual dimensions of the space where they are being displayed to improve loading time. Server-side rendering can improve your SEO altogether, but it is worth checking the size of the index.html file because sometimes it can be counterproductive. It is recommended to try and keep HTML files under 100kb to keep the TTFB (Time To First Byte) metric under 1 second. If the page uses polyfills, it's worth checking what polyfills are in use. IE11 is no longer supported, and loading unnecessary polyfills for that browser slows down the page load time. Performance Insights Tab The performance Insights Tab in Chrome DevTools allows users to measure the page load of a website. This is done by running a performance analysis on the website and providing metrics on various aspects of the page load process, such as the time it takes for the page to be displayed, the time it takes for network resources to be loaded, and the time it takes for the page to be interactive. The performance analysis is run by simulating a user visiting the website and interacting with it, which allows the tool to accurately measure the performance of the page under real-world conditions. This information can then be used to identify areas of the website that may be causing slowdowns and to optimize the performance of the page. Follow the steps to run an analysis: 1. Open the Chrome Devtools 2. Select the "Performance insights" tab 3. Click on the Measure page load button The analysis provides us with a detailed waterfall representation of requests, color coded to the request types. It can help you identify requests that block/slow down the page rendering, and/or expensive function calls that block the main thread. It also provides you with information on important performance metrics, such as DCL (DOM Content Loaded), FCP (First Contentful Paint), LCP (Largest Contentful Paint) and TTI (Time To Interactive). You can also simulate network or CPU throttling, or enable the cache if your use case requires that. DCL refers to the time it takes for the initial HTML document to be parsed and for the DOM to be constructed. FCP refers to the time it takes for the page to display the first contentful element, such as an image or text. LCP is a metric that measures the loading speed of the largest element on a webpage, such as an image or a block of text. A fast LCP helps ensure that users see the page's main content as soon as possible, which can improve the overall user experience. TTI refers to the time it takes for the page to become fully interactive, meaning that all of the necessary resources have been loaded and the page is responsive to user input. Performance Tab The "start profiling and reload page" button in the Performance tab of Chrome DevTools allows users to run a performance analysis on a website and view detailed information about how the page is loading and rendering. By clicking this button, the tool will simulate a user visiting the website and interacting with it, and will then provide metrics and other information about the page load process. Follow the steps to run an analysis 1. Open the Chrome Devtools 2. Select the "Performance" tab 3. Click on the button with the "refresh" icon A very useful part of this view is the detailed information provided on the main thread. We can interact with call stacks and find functions that might run too long, blocking the main thread, and delaying the TTI (Time To Interactive) metric. Selecting a function gives all kinds of information on that function. You can see how long that function was running, and what other functions it called, and you can also directly open that function in the Sources tab. Identifying long-running, blocking functions is crucial in finding performance bottlenecks. One way to mitigate them is to move them into worker threads. --- Chrome DevTools is a powerful tool for analyzing the performance of web applications. By using the network tab, you can identify issues with resources that might slow down page load. With the Performance insights and Performance tabs, we can identify issues that may be causing the website to load slowly, and take steps to optimize the code for better performance. Whether you're a beginner or an experienced developer, Chrome DevTools is an essential tool for analyzing and improving the performance of web applications....

Jan 9, 2023

6 mins

DevToolsSEOWeb PerformanceWeb Development

How to Style Using SCSS in Nuxt

Introduction SASS makes working on large projects more organized. It allows you to use variables, nested rules, mixins, and functions. The preferred styling method in Nuxt is component file styling, and integrating SASS into your project can make your component file styling appear more understandable. How to Import SASS in Nuxt To add SASS after setting up your Nuxt application, we will first install SASS and sass-loader. Let's run either of these commands depending on our package manager. ` Component File Styling With SASS and sass-loader installed in our project, we can now write SCSS in our component file. Lets see an example: ` In the example above, all we need to specify is lang="scss" on the style tag, and we can now write scss in that component. Global File Import and Variables To import SCSS files that are global, like variables and mixins files, we need to install style-resources: ` Next, we can update our nuxt.config.js file by adding the module we just installed to the buildModules. ` Next, let's create a global variables.scss file in our assets/style folder. Add a single variable inside: ` Next, we need to import this file inside the nuxt.config file: ` Now we have the variables in our variables.scss available in all our components for use. Next, let's test it out by updating our button component. ` We have updated our button variant color to be our global SCSS variable ($primary and $white). Mixins Mixins in SASS are used to write styles that can be reused in other parts of the code. Let's create a sample mixin to center an item. ` Next, we need to import our mixin in Nuxt config: ` Now, let's update our button component with our mixin: ` Functions Functions in SASS are used to write complex operations or behaviours that can be reused in other parts of the code. Let's create a function to handle a media query for our application. ` This is a basic example of what a function can be used for. More complex cases, like calculating percentage, can also be done. Let's import our mixin in Nuxt config: ` Let's update our button component with our function: ` We have been able to add SASS to our Nuxt project, and also looked at some ways in which SASS can make our codebase look cleaner. I hope this article has been helpful to you. If you encounter any issues, you can reach out to me on Twitter or Github....

Aug 17, 2022

2 mins

Web DevelopmentNuxtJS

From Coding Lonely to DevRel Ceremony - My Apprenticeship Journey

👎  Nervous, confused, belittled, defeated. These words describe the beginning of my professional coding journey. 👍  Empowered, excited, determined, appreciated. These words describe how I feel now, at the end of my apprenticeship. This sounds like an impossible 180, doesn't it? Let me tell you about this wild journey, and the things I learned along the way. 🐣  Where it all began.. At some point in our careers, we've been on that rollercoaster of excitement that quickly mixes with fear and doubt. Maybe you just started a new job, transitioned careers, or perhaps you're on that rollercoaster _RIGHT NOW_. 🎢  I felt this intensely when I was in my first dev job - I just moved to a new city, graduated from coding boot camp, and immediately started in the tech field within four months. Talk about moving fast! 💨 Fast forward seven months into my first dev job, and I found myself terrified of asking for help, staring at bugs with no support or solutions, and crying every night from the frustration. 😪  Although there were people who were sympathetic to my situation, I didn't feel good about myself or my job. I think we've all worked at places where we didn't feel supported, so I'll spare you the details. Bottom line - I realized that this environment was not conducive to my learning, and my confidence had broken down into rubble. 🤷‍♀️  Where was this person who was so eager and excited to learn? 🤔  Where was this person who was determined to find solutions, and contribute to the team? These were questions with answers that I didn't like, and didn't want to accept. So, I finally scrounged up the courage to quit. I FELT SO MUCH RELIEF! 😮‍💨  I didn't realize this at the time, but this was my form of self-care. To prioritize my health, and to *stop* the abuse. This new feeling of freedom felt so foreign to me. It was almost scary. But, I knew I broke the cycle, and I felt like I could breathe again. 🫁 _My Stress_ by NF speaks to the internal turmoil I felt during this time. _(".. Internal turmoil?" I know it sounds dramatic, but this period of my life was.. Rough.)_ .... So, what now? 🦗 🤷‍♀️  Why an apprenticeship? I wanted to code professionally, and stumbled upon This Dot's apprentice program, which was geared towards women in tech. This felt like a good mix of getting support as a junior dev, but also exploring opportunities to help others in underrepresented groups. I felt hesitant about an apprenticeship at first. 😰  I felt embarrassed to go from a junior dev position to being an apprentice. Of course, the clouds of doubt started reigning over. What if I can't fulfill the job? What if I look stupid? But when I started at This Dot, that attitude changed _REAL_ quick. I was welcomed with open arms by people who didn't even know me yet. In the back of my mind, I felt... suspicious. Why are they being so nice? _What are they buttering me up for?_ 🧈  My past experience wouldn't let me trust the positivity that I witnessed in this workspace. But in this new position, I had access to a whole new team, including senior engineers who had been through what I've been through and more. Not only was I able to work on coding projects with various clients - I was able to experience a whole new role in tech called DevRel (aka developer relations). And when I tell you that the work took off, _IT_ 👏  _TOOK_ 👏  _OFF_. As an apprentice, I had the flexibility to try out different projects. Within DevRel, I was able to write blogs, create Twitter content, and host This Dot events. But most importantly, I worked with a team that treated me with respect. I was afraid to veer away from 100% coding because it was all I knew, but I gained so much fulfillment from writing technical content, and engaging with the dev community. I was openly curious about how to write better content, and asked ALL the questions! I felt like I was learning, contributing, and engaging with the dev community in some manner, and it felt like I won the jackpot. 🥳 There was no longer any doubt. There was no question that this is where I was meant to be. I knew what it felt like to be respected at my job. I knew what it felt like to contribute something meaningful to my projects. (Side note: _From the bottom of my heart, I hope you've celebrated your wins, and experienced this type of enthusiasm and growth in your career. If you haven't yet, don't worry - it'll come._ ❤️ ) 🗺  Where am I now? After eight months in the apprentice program, I'm happy to write that I'll be starting my permanent full-time role as a Community Engineer 💻  at StackBlitz. This role will allow me to continue engaging with the community, and leading internal and external projects to improve the community's experience. This role encompasses tasks along the DevRel side that make me even more excited to be a part of this community. Like This Dot, StackBlitz has welcomed me with open arms, and I'm so excited to work with a great team. 🏅 📝  Tips I learned 1. Be Explicit 🗣 Nothing can harm a project or team more than ambiguity. Ambiguity can cause misunderstandings in objectives, delays in progress, and miscommunication in your team. Especially when you're starting out, be explicit in the support you need. Be explicit when you're outlining the work you're doing. When you're talking to your manager or mentor about professional goals, be explicit about what you want! Remember, we are not mind readers. 😊 2. Find Your A-Team 📣 There's no cookie-cutter outline in what this looks like. It all depends on what YOU need! Find a group of people who truly support you, whether it's your manager, a coworker, or a mentor outside of your company. Find the people who will celebrate your wins as their own. Find the people who won't be afraid to call you out when you're in the wrong. Find people who are invested in your growth just as much as you are. 🌱 3. Get Out of Your Head 🧠 When we're too inside of our heads, we may unintentionally inflate situations that make it harder to construct realistic resolutions. I find it so helpful (and therapeutic 💆🏻‍♀️  ) to write things out. Whether it's in my physical bullet journal, Notion, a sticky note, or just talking to someone - Getting the thoughts out of my head allowed me to understand the situation at hand, and outline what I needed to do next. Our minds are prime real estate 🏠  so choose wisely what you allow to live rent-free. 😉 (Admittedly, I allow this dog to live rent-free because he's so stinking adorable. 🐶  ) 4. Show Empathy & Be Compassionate 🤗 These can be shown in big or small ways, and every bit matters. Sometimes that can be adding resilient tests, and writing code that other developers can easily understand and maintain. Sometimes, that can be contributing well-written documentation that's accessible to your user community. And sometimes it can look like asking someone how their day has been. My favorite way of doing this is celebrating wins, and giving kudos to others because when one of us wins, we all win. 🎉  We're humans, not machines! 🤖  So let's treat each other as such. 💜 🎬  Conclusion I know this was a lengthy read, and I appreciate you taking the time to read about my journey! I'm very fortunate to have found people who support me and to be in a position to acknowledge my growth. I encourage "Code Newbies" to consider an apprenticeship as part of their developer journey. Kudos to This Dot for their evolving apprentice program, and a big "thank you" to StackBlitz for allowing me to continue my tech journey with them! 🚙...

Jul 20, 2021

7 mins

General

Introduction to Zod for Data Validation

As web developers, we're often working with data from external sources like APIs we don't control or user inputs submitted to our backends. We can't always rely on this data to take the form we expect, and we can encounter unexpected errors when it deviates from expectations. But with the Zod library, we can define what our data ought to look like and parse the incoming data against those defined schemas. This lets us work with that data confidently, or to quickly throw an error when it isn't correct. Why use Zod? TypeScript is great for letting us define the shape of our data in our code. It helps us write more correct code the first time around by warning us if we are doing something we shouldn't. But TypeScript can't do everything for us. For example, we can define a variable as a string or a number, but we can't say "a string that starts with user_id_ and is 20 characters long" or "an integer between 1 and 5". There are limits to how much TypeScript can narrow down our data for us. Also, TypeScript is a tool for us developers. When we compile our code, our types are not available to the vanilla JavaScript. JavaScript can't validate that the data we actually use in our code matches what we thought we'd get when we wrote our TypeScript types unless you're willing to manually write code to perform those checks. This is where we can reach for a tool like Zod. With Zod, we can write data schemas. These schemas, in the simplest scenarios, look very much like TypeScript types. But we can do more with Zod than we can with TypeScript alone. Zod schemas let us create additional rules for data parsing and validation. A 20-character string that starts with user_id_? It's z.string().startsWith('user_id_').length(20). An integer between 1 and 5 inclusive? It's z.number().int().gte(1).lte(5). Zod's primitives give us many extra functions to be more specific about *exactly* what data we expect. Unlike TypeScript, Zod schemas aren't removed on compilation to JavaScript—we still have access to them! If our app receives some data, we can verify that it matches the expected shape by passing it to your Zod schema's parse function. You'll either get back your data in exactly the shape you defined in your schema, or Zod will give you an error to tell you what was wrong. Zod schemas aren't a replacement for TypeScript; rather, they are an excellent complement. Once we've defined our Zod schema, it's simple to derive a TypeScript type from it and to use that type as we normally would. But when we really need to be sure our data conforms to the schema, we can always parse the data with our schema for that extra confidence. Defining Data Schemas Zod schemas are the variables that define our expectations for the shape of our data, validate those expectations, and transform the data if necessary to match our desired shape. It's easy to start with simple schemas, and to add complexity as required. Zod provides different functions that represent data structures and related validation options, which can be combined to create larger schemas. In many cases, you'll probably be building a schema for a data object with properties of some primitive type. For example, here's a schema that would validate a JavaScript object representing an order for a pizza: ` Zod provides a number of primitives for defining schemas that line up with JavaScript primitives: string, number, bigint, boolean, date, symbol, undefined, and null. It also includes primitives void, any, unknown, and never for additional typing information. In addition to basic primitives, Zod can define object, array, and other native data structure schemas, as well as schemas for data structures not natively part of JavaScript like tuple and enum. The documentation contains considerable detail on the available data structures and how to use them. Parsing and Validating Data with Schemas With Zod schemas, you're not only telling your program what data should look like; you're also creating the tools to easily verify that the incoming data matches the schema definitions. This is where Zod really shines, as it greatly simplifies the process of validating data like user inputs or third party API responses. Let's say you're writing a website form to register new users. At a minimum, you'll need to make sure the new user's email address is a valid email address. For a password, we'll ask for something at least 8 characters long and including one letter, one number, and one special character. (Yes, this is not really the best way to write strong passwords; but for the sake of showing off how Zod works, we're going with it.) We'll also ask the user to confirm their password by typing it twice. First, let's create a Zod schema to model these inputs: ` So far, this schema is pretty basic. It's only making sure that whatever the user types as an email is an email, and it's checking that the password is at least 8 characters long. But it is *not* checking if password and confirmPassword match, nor checking for the complexity requirements. Let's enhance our schema to fix that! ` By adding refine with a custom validation function, we have been able to verify that the passwords match. If they don't, parsing will give us an error to let us know that the data was invalid. We can also chain refine functions to add checks for our password complexity rules: ` Here we've chained multiple refine functions. You could alternatively use superRefine, which gives you even more fine grained control. Now that we've built out our schema and added refinements for extra validation, we can parse some user inputs. Let's see two test cases: one that's bound to fail, and one that will succeed. ` There are two main ways we can use our schema to validate our data: parse and safeParse. The main difference is that parse will throw an error if validation fails, while safeParse will return an object with a success property of either true or false, and either a data property with your parsed data or an error property with the details of a ZodError explaining why the parsing failed. In the case of our example data, userInput2 will parse just fine and return the data for you to use. But userInput1 will create a ZodError listing all of the ways it has failed validation. ` ` We can use these error messages to communicate to the user how they need to fix their form inputs if validation fails. Each error in the list describes the validation failure and gives us a human readable message to go with it. You'll notice that the validation errors for checking for a valid email and for checking password length have a lot of details, but we've got three items at the end of the error list that don't really tell us anything useful: just a custom error of Invalid input. The first is from our refine checking if the passwords match, and the second two are from our refine functions checking for password complexity (numbers and special characters). Let's modify our refine functions so that these errors are useful! We'll add our own error parameters to customize the message we get back and the path to the data that failed validation. ` Now, our error messages from failures in refine are informative! You can figure out which form fields aren't validating from the path, and then display the messages next to form fields to let the user know how to remedy the error. ` By giving our refine checks a custom path and message, we can make better use of the returned errors. In this case, we can highlight specific problem form fields for the user and give them the message about what is wrong. Integrating with TypeScript Integrating Zod with TypeScript is very easy. Using z.infer<typeof YourSchema> will allow you to avoid writing extra TypeScript types that merely reflect the intent of your Zod schemas. You can create a type from any Zod schema like so: ` Using a TypeScript type derived from a Zod schema does *not* give you any extra level of data validation at the type level beyond what TypeScript is capable of. If you create a type from z.string.min(3).max(20), the TypeScript type will still just be string. And when compiled to JavaScript, even that will be gone! That's why you still need to use parse/safeParse on incoming data to validate it before proceeding as if it really does match your requirements. A common pattern with inferring types from Zod schemas is to use the same name for both. Because the schema is a variable, there's no name conflict if the type uses the same name. However, I find that this can lead to confusing situations when trying to import one or the other—my personal preference is to name the Zod schema with Schema at the end to make it clear which is which. Conclusion Zod is an excellent tool for easily and confidently asserting that the data you're working with is exactly the sort of data you were expecting. It gives us the ability to assert at runtime that we've got what we wanted, and allows us to then craft strategies to handle what happens if that data is wrong. Combined with the ability to infer TypeScript types from Zod schemas, it lets us write and run more reliable code with greater confidence....

Nov 15, 2024

7 mins

JavaScriptZod

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

#StateOfAutomation - Simple Solutions for Complex Problems

👩‍💻 Alison Dowdney (@alisondowdney)

Developer Advocate from Weaveworks

🧑‍💻 Taylor Dolezal (@onlydole)

Senior Developer Advocate from HashiCorp

🧑‍💻 Zan Markan (@zmarkan)

Developer Advocate from CircleCI

👩‍💻 Tracy Miranda (@tracymiranda)

Executive Director from Continuous Delivery Foundation

🔑 Key Takeaway Points

🧠 Resource Recommendations

Susan Ma

You might also like

Performance Analysis with Chrome DevTools

How to Style Using SCSS in Nuxt

From Coding Lonely to DevRel Ceremony - My Apprenticeship Journey

Introduction to Zod for Data Validation

Let's innovate together!

You might also like

Performance Analysis with Chrome DevTools

How to Style Using SCSS in Nuxt

From Coding Lonely to DevRel Ceremony - My Apprenticeship Journey

Introduction to Zod for Data Validation