Javascript

Integrating Next.js with New Relic

Published Jul 19, 2024

Updated Jul 19, 2024

7 min read

Integrating Next.js with New Relic

When it comes to application monitoring, New Relic is genuinely one of the veterans in the market. Founded in 2008, it offers a large set of tools designed for developers to help them analyze application performance, troubleshoot problems, and gain insight into user behavior through real-time analytics.

We needed to integrate New Relic's application monitoring with a modern Next.js app built on top of the app router. This blog post will explain your options if you want to do the same in your project. Please keep in mind that depending on when you read this blog post, there may be some advances in how New Relic works with Next.js. You can always refer to the official New Relic website for up-to-date documentation.

The approach is also different, depending on whether you use Vercel's managed Next.js hosting (or any other provider that is lambda-based) or self-host Next.js on an alternative provider such as Fly.io. We'll start with the typical steps you need to do for both ways of deploying; then we'll continue with the self-hosted option, which uses a complete set of features from New Relic, and finish with Vercel where we recommend the OpenTelemetry integration for now.

Common Prerequisite Steps

First, let's install the necessary packages. These were built by New Relic and are needed whether you self-host your Next.js app or host it on Vercel.

npm install newrelic @newrelic/next
npm install -D @types/newrelic

The next thing you'll want to do is define your app name. The name will be different depending on the environment the app is running in, such as a local environment, staging, or production. This will ensure that the data you send to New Relic is always tagged with a proper app name and will not be mixed between environments.

An environment-changing parameter, such as the app name, should always be included in an environment variable. In your local environment, that would be in .env.local, while staging/production or any other hosted environments would be configured in the provider.

NEW_RELIC_APP_NAME='My App - Staging'

To get full coverage with New Relic, in Next.js, we need to cover the parts running on the client (in the browser) and on the server. The server-running part is different on Vercel's managed hosting, where all of the logic is executed within lambdas, and self-hosting variants, where the server logic is executed on a Node.js server.

The client part is configured by going to the New Relic dashboard and clicking "Add Data" / "Browser Monitoring". Choose "place a JavaScript snippet in frontend code," name it the same name you used in the environment variable above, click "Continue" to the last step, and copy the part between the script tags:

<script type="text/javascript">
	// big chunk of JavaScript that you need to copy
</script>

Then paste it to a variable inside any module of your code. For example, this is how we did it:

// app/components/new-relic-scripts-per-env.ts

export const DEV_SCRIPT=`script contents`;
export const STAGING_SCRIPT=`script contents`;
export const PRODUCTION_SCRIPT=`script contents`;

Now, create a component that will use this script and put it to Next's Script component. Then you can use the component in your root layout:

// app/components/new-relic-script.tsx
import Script from "next/script";
import * as React from "react";
import { IS_PRODUCTION, IS_STAGING } from "@/utilities/environment";
import {
  DEV_SCRIPT,
  PRODUCTION_SCRIPT,
  STAGING_SCRIPT,
} from "./new-relic-scripts-per-env";

function getScriptContent() {
  if (IS_PRODUCTION) {
    return PRODUCTION_SCRIPT;
  }

  if (IS_STAGING) {
    return STAGING_SCRIPT;
  }

  return DEV_SCRIPT;
}

export default function NewRelicScript() {
  const scriptContent = getScriptContent();

  return (
    <Script
      id="nr-browser-agent"
      strategy="beforeInteractive"
    >
      {scriptContent}
    </Script>
  );
}

There are several things to note here. First, we need to set a unique ID for the Script component for Next.js to track and optimize the script. We also need to set the strategy to beforeInteractive so that the New Relic script is added to the document’s head element. Finally, we use some environment helper variables. IS_PRODUCTION and IS_STAGING are just helper variables that read the environment variables to determine if we are in a local environment, staging, or production.

Now you can create an app router error page and report any uncaught errors to New Relic:

// app/error.tsx
"use client";

import { useEffect } from "react";

export default function ErrorPage({
  error,
}: {
  error: Error & { digest?: string };
}) {
  useEffect(() => {
    if (window.newrelic) {
      window.newrelic.noticeError(error);
    } else {
      console.error("Cannot report error to New Relic", error);
    }
  }, [error]);

  return (
    <div>
      <h2>Something went wrong!</h2>
    </div>
  );
}

With this in place, the client part is covered, and you should start seeing data in the New Relic dashboard under "All Entities" / "Browser Applications."

Let's now focus on the server part, with both a self-hosted option and Vercel.

Self-Hosting Next.js

If you self-host Next.js, It will typically run through a Node server (containerized or not), meaning you can fully utilize New Relic's APM agent.

Start by adding this to the Next.js config (next.config.js) to mark the newrelic package as external, otherwise you would not be able to import it.

{
  ...
  experimental: {
    ...
    // Without this setting, the Next.js compilation step will routinely
    // try to import files such as `LICENSE` from the `newrelic` module.
    // Note that this property will be renamed in the future: https://github.com/vercel/next.js/pull/65421
    serverComponentsExternalPackages: ["newrelic"],
  },
}

The next step that New Relic recommends is to copy the newrelic.js file from the newrelic library to the root of the folder. However, we want to keep our project root clean, and we'll store the file in o11y/newrelic.js (“o11y” is an abbreviation for observability). This will also demonstrate the configuration needed to make this possible.

This file, by default, will look something like this.

"use strict";

/**
 * New Relic agent configuration.
 *
 * See lib/config/default.js in the agent distribution for a more complete
 * description of configuration variables and their potential values.
 */
exports.config = {
  /**
   * Array of application names.
   */
  app_name: [process.env.NEW_RELIC_APP_NAME],
  /**
   * Your New Relic license key.
   */
  license_key: process.env.NEW_RELIC_LICENSE_KEY,
  agent_enabled: true,
  logging: {
    /**
     * Level at which to log. 'trace' is most useful to New Relic when diagnosing
     * issues with the agent, 'info' and higher will impose the least overhead on
     * production applications.
     */
    level: "info",
  },
  /**
   * When true, all request headers except for those listed in attributes.exclude
   * will be captured for all traces, unless otherwise specified in a destination's
   * attributes include/exclude lists.
   */
  allow_all_headers: true,
  attributes: {
    /**
     * Prefix of attributes to exclude from all destinations. Allows * as wildcard
     * at the end.
     *
     * NOTE: If excluding headers, they must be in camelCase form to be filtered.
     *
     * @name NEW_RELIC_ATTRIBUTES_EXCLUDE
     */
    exclude: [
      "request.headers.cookie",
      "request.headers.authorization",
      "request.headers.proxyAuthorization",
      "request.headers.setCookie*",
      "request.headers.x*",
      "response.headers.cookie",
      "response.headers.authorization",
      "response.headers.proxyAuthorization",
      "response.headers.setCookie*",
      "response.headers.x*",
    ],
  },
};

As you can see, some environment variables are involved, such as NEW_RELIC_APP_NAME and NEW_RELIC_LICENSE_KEY. These environment variables will be needed when the New Relic agent starts up. The trick here is that the agent doesn't start up with your app but earlier by preloading the library through another environment variable, NODE_OPTIONS.

On the server, that would be done by modifying the start script to this:

{
  "scripts": {
    ...
    "start": "NODE_OPTIONS='--loader newrelic/esm-loader.mjs -r @newrelic/next --no-warnings' next start",
  }
}

This ensures the agent is preloaded before starting the app, and the rest of the environment variables (such as NEW_RELIC_APP_NAME and NEW_RELIC_LICENSE_KEY) should already exist. Making sure this works in local development is not that easy due to different stages of loading environment variables. The environment variables from .env.local are loaded by the framework, which is too late for the New Relic agent (which is preloaded by the Node process). However, it is possible with a trick.

First, add the following new environment variables to .env.local:

NEW_RELIC_LICENSE_KEY=# take from 1Password
NEW_RELIC_HOME=o11y
# NODE_OPTIONS is needed only for local development
NODE_OPTIONS='--loader newrelic/esm-loader.mjs -r @newrelic/next --no-warnings'

(We have not mentioned NEW_RELIC_HOME so far, but the New Relic agent implicitly uses it to locate the newrelic.js file. We need to specify that since we moved it to the o11y folder.)

Next, install the nodenv package.

npm install -D node-env-run

The nodenv package provides a cross-platform way of preloading environment variables before we start the Node process. If we provide it with the .env.local file, this means all of the environment variables from that file will be available to the New Relic agent!

Change the scripts in package.json:

{
  "scripts": {
    "dev": "nodenv -E .env.local --exec \"next dev --turbo -H apollo.knopmanmarks.localhost -p 3000\"",
    "start": "NODE_OPTIONS='--loader newrelic/esm-loader.mjs -r @newrelic/next --no-warnings' next start",
  }
}

And that's it. If you now start the local server using the dev command, nodenv will make environment variables available to the Node process. The Node process will preload the New Relic agent (because we have NODE_OPTIONS in .local.env), the agent will read all the environment variables needed by newrelic.js, and everything will work as expected.

The first thing you might notice is the creation of the newrelic_agent.log. This is just a log file by the agent which you can use to troubleshoot if the agent is not working correctly. You can safely add this file to .gitignore.

From this point on, the server-side part of the app can utilize New Relic's API (such as noticeError and recordLogEvent) and the agent will automatically be recording transactions made through the server. The data should be visible in the New Relic dashboard, under "All Entities" / "Services - APM".

This is how you would log an error in your server action, for example:

import newrelic from "newrelic";

export async function createUserAction(...) {
  try {
    // create user logic
  } catch (err) {
    newrelic.noticeError(err);
  }
}

The full source of this type of integration is available on StackBlitz.

Using the New Relic APM agent on Vercel is currently not possible, as the server part of Next.js runs within lambdas, and not within a typical Node server as in the self-hosted option. In the future, this may be possible. At the moment, the only way to integrate Vercel and New Relic is by using the Vercel OpenTelemetry collector, which is only available for some plans.

The first step is to enable the integration in Vercel's dashboard and then install the following packages:

npm install @vercel/otel @opentelemetry/api

Create an instrumentation.ts file in the root of your project and add the following code:

import { registerOTel } from "@vercel/otel";
import { NEW_RELIC_APP_NAME } from "@/utilities/environment";

registerOTel({ serviceName: NEW_RELIC_APP_NAME });

In the the Next.js config (next.config.js), add:

{
  ...
  experimental: {
    instrumentationHook:
      !!process.env.VERCEL_ENV && process.env.VERCEL_ENV !== "development", // Only use instrumentation if deployed to Vercel
  },
}

The OpenTelemetry integration does not work locally, hence the above conditional check. With the above configuration, the collected OpenTelemetry data should be visible in the New Relic dashboard under "All Entities" / "Services - OpenTelemetry".

Optionally, Logging via New Relic API

When using the Vercel OpenTelemetry integration, there is no easy way to submit custom logs to New Relic. New Relic offers a logging API, however, so to send custom logs from your backend, you could use a logger such as Winston, which has good library support for custom transports, such as the one for New Relic.

Install the following packages:

npm install winston winston-nr

Then create a logger.ts file with the following code:

import {
  IS_VERCEL,
  LOG_LEVEL,
  NEW_RELIC_APP_NAME,
  NEW_RELIC_LICENSE_KEY,
} from "./environment";
import { headers } from "next/headers";
import winston from "winston";
import { NewRelicTransport } from "winston-nr";

const newRelicTransport = new NewRelicTransport({
  apiUrl: "https://log-api.newrelic.com/log/v1",
  apiKey: NEW_RELIC_LICENSE_KEY,
  compression: true,
  retries: 3,
  batchSize: 10,
  batchTimeout: 5000,
  format: winston.format.json(),
});

export const serverLogger = winston.createLogger({
  level: LOG_LEVEL,
  defaultMeta: { serviceName: NEW_RELIC_APP_NAME },
  transports: [newRelicTransport],
});

NewRelicTransport utilizes the New Relic logging API to send application logs. Now you can use the serverLogger in your backend code, from server components to server actions.

Conclusion

As you can see, various options exist for integrating New Relic with the modern, app-router-based Next.js. Sadly, at the time of writing, no one-size-fits-all solution works equally for all hosting options, as Vercel's managed Next.js platform is still relatively young, and so is the app router in Next.js.

If you liked this blog post, feel free to browse our other Next.js blog posts for more tips and tricks in other areas of the Next.js ecosystem.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

Dario Djuric
Dario is a full-stack engineer who has spent most of his career doing enterprise Java projects. He has always had a hidden passion for frontend, though -- and he is now able to pursue that passion at This Dot. He spends most of his free time with his two sons, but occasionally, he manages to squeeze in some casual sports activities such as jogging, cycling, and soccer.
@dario_djuric @dariodjuric

Tim Neutkens, Co-Author of Next.js on the State of Next

Watch this episode! Tim Neutkens, Co-author and Tech Lead for Next.js, discusses how open source maintainers are simplifying the web, and covers the challenges faced with the current Next.js setup. Tim talks about TurboPack, a solution that optimizes bundling, improves parallelism, caching, and module graph calculations. He also talks about TurboAC, which focuses on addressing performance and compatibility issues, providing seamless transitions for Next.js users. Tim highlights the importance of efficient bundling processes to avoid excessive recompilation and discusses the updates in Next.js versions to enhance caching, rendering behavior, and client-side caching. Tim also discusses some exciting upcoming features in Next.js 15. Tim Neutkens' Socials - Twitter: @timneutkens - GitHub: timneutkens - Bluesky: timneutkens.bsky.social - Website: https://timn.tech/ Links - Vercel on Twitter, LinkedIn, Facebook, Instagram, YouTube, GitHub and Vercel's website - Next.js on Twitter, GitHub, LinkedIn, YouTube, Instagram, Facebook, official Next.js website - Turbopack on Twitter, GitHub, YouTube, LinkedIn, Instagram, Official Turbopack Docs - Webpack on GitHub, Twitter, YouTube, and Official Webpack Website Show Notes - [00:00:02] Podcast episode featuring Tim Neutkens discussing Next.js and the upcoming release of TurboPack. - [00:04:27] JavaScript bundlers evolving to handle growth. - [00:07:58] TurboPack solves Webpack limitations efficiently. - [00:12:12] Bundler compatibility for optimal app performance. - [00:16:50] Client components separated in webpack instance. Turbo pack for better parallelism and stability. Industry moving towards server-side. Feed and rollup still relevant. Collaboration between tools for future. - [00:20:57] Replacing part with roll down, similar to Webpack. Overlapping ecosystem with Avonetic Conference. Limits with unbundling and loading on demand. Cycle of building frameworks and hitting limits. History of using Webpack for client-side code. Two compiler architecture for server and client. Coordination between server and client with Webpack. - [00:25:38] Server action imports, turbo pack improves performance. - [00:30:04] Next.js is popular for websites. - [00:34:18] Chipotle using Next in Vercel, exciting improvements. - [00:38:51] Next.js 15 release candidate with changes. This episode is sponsored by Wix Studio....

Aug 7, 2024

2 mins

VercelNextJS

Maximizing Server Rendering for Interactive Next.js Applications

Maximizing Server Rendering for Interactive Next.js Applications Next.js has come a long way since its inception, and it's now a powerful framework that combines both server-side and client-side rendering. With the introduction of React Server Components (RSCs) and the classic client components, you have the flexibility to create a purely client-rendered app with many client components or a purely server-rendered app using RSCs. However, what if you want to have the best of both worlds? Server rendering improves performance and can deliver much of the HTML quickly, reducing the need for the client to render the entire page. This also results in smaller bundles delivered to the client. The question is, how would you maximize server rendering while still having an interactive application? This blog post will show you how. Building a Lightweight Shopping Experience To illustrate the concept, we'll build a lightweight shopping experience with a list of products and an "Add to Cart" button that opens a drawer with the added cart items. Most of the page will be server-rendered, and we'll use client components only to show the drawer. Even the drawer itself will not be entirely a client component; rather, the wrapper necessary to show the drawer will be a client component, while the contents will be server-rendered. This can be accomplished by the nature of client components that can accept server-rendered content. We'll set up a React context provider at the layout level, which will control showing or hiding the cart. This provider will be used by our client components (the "Add to Cart" button and the "Show Cart" button), as well as by the drawer wrapper. When you click "Add to Cart," this will invoke a server action (more on that later), and then use a function provided through the context of the provider to show the drawer. The Code For this example, we'll use shadcn as our component library. It looks nice, and it also provides a good, accessible component for our cart drawer. Looking at our wireframe above, we'll need a total of three components: - Button - Table - Sheet The components can be installed with a single command: ` Next, we'll set up our React context provider: ` ` The provider provides a boolean flag to indicate whether the cart is currently shown (isCartShown) and a function to change that (setCartShown). Note that CartProvider is a client component, but its contents are provided as children. Therefore, CartProvider is just a thin client-component wrapper around its contents. Now, let's see the implementation of the "Add to Cart" button and the CartDrawer component. The AddToCartButton is a client component that reads the React context and calls its setCartShown function after adding to the cart has been completed. This will, in turn, set the isCartShown flag on the provider and show the cart drawer, which also reads the isCartShown flag from the provider. How is the cart updated, though? We'll use a server action named addToCartAction for that. We'll explain how this works in a bit, but in the meantime, let's see the implementations of the AddToCart button and the CartDrawer component. ` Similar to the CartProvider component, the' CartDrawer' component is a thin client component wrapper around its contents. It reads the React context provided by the CartProvider and passes it to the Sheet component from the Shadcn library. The SheetContent component accepts cart contents as children passed from the CartDrawer's parent, which is server-rendered. ` Using Server Actions and Cache Invalidation But how do we ensure that the contents are updated? The contents are an RSC component called CartContents that calls getCachedCart in the data access layer. In real-world scenarios, getCachedCart would probably call a database or an API, but we have neither here. So, we'll simulate a database through the use of the file-system-db package, a lightweight Node library that can write and read to a JSON file as a database. The get calls to the database are wrapped in unstable_cache, which allows us to tag such requests with cart. This means CartContents is initially fetching data from the tagged getCachedCart call. ` Whenever we mutate the cart, we want to call invalidateTags("cart"). This will force purging of Next.js caches and consequently re-render the cart. For example, the "Add to Cart" button calls the addToCartAction server action, which adds the item to the database and calls invalidateTags("cart"). This server action call returns an updated RSC payload to the client, which re-renders the part that depended on the getCachedCart call - and that is the CartContents component. ` ` And that's it. We now have a fully functional, mostly server-rendered cart, where only small bits of interactivity are modeled as client components. Here is the result: Some of the code shown above remains, such as the "Show Cart" button and the table of products, but those are just UX improvements on top of the core functionality. You can see the entire source code on Stackblitz. Wrapping Up In this blog post, we've explored how to maximize server rendering in Next.js while still having an interactive application. By leveraging React Server Components, client components, and server actions, we've created a shopping experience where most of the page is server-rendered, but the cart drawer is interactive and updates in real time. This approach improves performance, reduces the client-side rendering burden, and delivers smaller bundles to the client....

May 15, 2024

5 mins

NextJS

A Look at Playwright Parallelism

A Look at Playwright Parallelism Playwright is an open-source automation library developed by Microsoft, designed for testing web applications across multiple browsers. It enables developers and testers to write scripts that simulate real user actions when interacting with web pages. Playwright supports major browsers like Chrome, Firefox, and Safari, including their headless modes. Recently, it has gained a lot of popularity for its robustness, ease of use, and cross-browser compatibility. In this blog post, we will take a look at one very useful aspect of Playwright: running tests in parallel. Parallelism in Playwright Parallelism in Playwright refers to running multiple test spec files or even test cases within a spec file simultaneously, greatly improving test execution speed. This is achieved by running the tests in worker processes, where each worker process is an OS process, running independently, orchestrated by the test runner. All workers have identical environments, and each starts its own browser instance. Parallel execution is particularly beneficial in continuous integration and continuous deployment (CI/CD) pipelines, reducing overall build and testing times. Playwright's architecture inherently supports parallelism, and most modern test runners integrating with Playwright can leverage this feature. The parallel execution feature makes Playwright a highly efficient tool for large-scale web application testing. Enabling Parallelism By default, if you scaffold the project using npm init playwright@latest, parallelism is already enabled. Assuming that Playwright's configuration includes three browsers and there is a single spec file with two test cases, the total number of tests that Playwright needs to execute is 3x2 = 6. ` Playwright will decide on how many workers to use based on several factors: - Hardware support: Depending on the number of CPU cores and other system resources, the operating system will decide how many processes Playwright can spin up. - The workers property in the playwright.config.ts file. - The --workers argument to the playwright test command. For example, npx playwright test --workers 4. The --workers argument overrides the workers property in the configuration. However, in both cases, the number of workers can go up to the number of processes that Playwright can spin up, as decided by the operating system. Once it has determined the number of workers, and if the number of workers is larger than 1, Playwright will then decide how to split the work between workers. This decision also depends on several factors, such as the number of browsers involved and the granularity of the parallelism, which can be controlled in several ways: - In the test spec file, you can specify whether to run the test cases in parallel. To run tests in parallel, you can invoke test.describe.configure({ mode: 'parallel' }); before your test cases. - Alternatively, you can configure it per project by setting the fullyParallel: true property. - And finally, you can set it globally in the config, using the same property: fullyParallel: true. Therefore, if there is more than one worker and the parallel mode is enabled, Playwright will assign test cases to each worker as they become available. This scenario is ideal because it means that each test is stateless, and the resources are used most efficiently. ` Disabling Parallelism What if, however, the tests are not stateless? Imagine one test changes the global configuration of the app via some sort of administration page, and the configuration affects different parts of the app, like enabling or disabling features. Other tests, which may be testing those features, would be impacted and would report incorrect results. In such cases, you might want to disable parallelism. You can disable any parallelism globally by allowing just a single worker at any time. Following the instructions from the previous sections to configure the number of workers, you can either set the workers: 1 option in the configuration file or pass --workers=1 to the command line. ` Let's have a look at our test output in this case: ` Now, compare the time it took with one worker versus the time it took with four workers. It took 8.2 seconds with one worker compared to only 3.9 seconds with multiple workers. That might be an inefficient usage of resources, of course, especially if you have a large test suite and some of the tests are stateless and can be run without impacting other tests. Tweaking Parallelism If you want some tests not to run in parallel, but still want to utilize your workers, you can do that. Again, following the configuration options from the previous sections, you can annotate the entire spec file with test.describe.configure({ mode: 'serial' }); to have the tests run sequentially in that spec file, or use the fullyParallel: false property to run tests sequentially on the project level, or using the same property to run tests sequentially on the global level. This means you can still split the tests between workers, but the tests would be run sequentially within a worker depending on the configuration. For example, let's set the number of workers to 4, but set fullyParallel: false globally. ` The tests need to be run sequentially, but since each browser instance effectively provides an isolated environment, this means tests cannot impact each other between environments, and they are safe to be executed sequentially within an environment. This means setting fullyParallel: false on the global level is not the same as having workers: 1, since the browsers themselves offer an isolated environment for the tests to run sequentially. However, since we only have 3 environments (3 browsers), we cannot fully utilize 4 workers as we wanted, so the number of workers is 3. Conclusion In conclusion, Playwright's workers are the core of its parallelism capabilities, enabling tests to run concurrently across different environments and browsers with efficiency. Through its many configuration properties, Playwright allows you to configure parallelism at multiple levels, offering a granular approach to optimizing your test runs. Beyond just executing tests in parallel on a single machine, Playwright's parallelism can even extend to splitting work across multiple machines through sharding, significantly enhancing the scalability of your testing. We hope this blog post was useful. For those interested in delving deeper into the world of Playwright and its powerful features, we've also recently released a JS Drop titled Awesome Web Testing with Playwright, and we also hosted Debbie O'Brien from Microsoft in a Modern Web episode....

Mar 20, 2024

5 mins

JavaScriptPlaywrightTesting

Introduction to Zod for Data Validation

As web developers, we're often working with data from external sources like APIs we don't control or user inputs submitted to our backends. We can't always rely on this data to take the form we expect, and we can encounter unexpected errors when it deviates from expectations. But with the Zod library, we can define what our data ought to look like and parse the incoming data against those defined schemas. This lets us work with that data confidently, or to quickly throw an error when it isn't correct. Why use Zod? TypeScript is great for letting us define the shape of our data in our code. It helps us write more correct code the first time around by warning us if we are doing something we shouldn't. But TypeScript can't do everything for us. For example, we can define a variable as a string or a number, but we can't say "a string that starts with user_id_ and is 20 characters long" or "an integer between 1 and 5". There are limits to how much TypeScript can narrow down our data for us. Also, TypeScript is a tool for us developers. When we compile our code, our types are not available to the vanilla JavaScript. JavaScript can't validate that the data we actually use in our code matches what we thought we'd get when we wrote our TypeScript types unless you're willing to manually write code to perform those checks. This is where we can reach for a tool like Zod. With Zod, we can write data schemas. These schemas, in the simplest scenarios, look very much like TypeScript types. But we can do more with Zod than we can with TypeScript alone. Zod schemas let us create additional rules for data parsing and validation. A 20-character string that starts with user_id_? It's z.string().startsWith('user_id_').length(20). An integer between 1 and 5 inclusive? It's z.number().int().gte(1).lte(5). Zod's primitives give us many extra functions to be more specific about *exactly* what data we expect. Unlike TypeScript, Zod schemas aren't removed on compilation to JavaScript—we still have access to them! If our app receives some data, we can verify that it matches the expected shape by passing it to your Zod schema's parse function. You'll either get back your data in exactly the shape you defined in your schema, or Zod will give you an error to tell you what was wrong. Zod schemas aren't a replacement for TypeScript; rather, they are an excellent complement. Once we've defined our Zod schema, it's simple to derive a TypeScript type from it and to use that type as we normally would. But when we really need to be sure our data conforms to the schema, we can always parse the data with our schema for that extra confidence. Defining Data Schemas Zod schemas are the variables that define our expectations for the shape of our data, validate those expectations, and transform the data if necessary to match our desired shape. It's easy to start with simple schemas, and to add complexity as required. Zod provides different functions that represent data structures and related validation options, which can be combined to create larger schemas. In many cases, you'll probably be building a schema for a data object with properties of some primitive type. For example, here's a schema that would validate a JavaScript object representing an order for a pizza: ` Zod provides a number of primitives for defining schemas that line up with JavaScript primitives: string, number, bigint, boolean, date, symbol, undefined, and null. It also includes primitives void, any, unknown, and never for additional typing information. In addition to basic primitives, Zod can define object, array, and other native data structure schemas, as well as schemas for data structures not natively part of JavaScript like tuple and enum. The documentation contains considerable detail on the available data structures and how to use them. Parsing and Validating Data with Schemas With Zod schemas, you're not only telling your program what data should look like; you're also creating the tools to easily verify that the incoming data matches the schema definitions. This is where Zod really shines, as it greatly simplifies the process of validating data like user inputs or third party API responses. Let's say you're writing a website form to register new users. At a minimum, you'll need to make sure the new user's email address is a valid email address. For a password, we'll ask for something at least 8 characters long and including one letter, one number, and one special character. (Yes, this is not really the best way to write strong passwords; but for the sake of showing off how Zod works, we're going with it.) We'll also ask the user to confirm their password by typing it twice. First, let's create a Zod schema to model these inputs: ` So far, this schema is pretty basic. It's only making sure that whatever the user types as an email is an email, and it's checking that the password is at least 8 characters long. But it is *not* checking if password and confirmPassword match, nor checking for the complexity requirements. Let's enhance our schema to fix that! ` By adding refine with a custom validation function, we have been able to verify that the passwords match. If they don't, parsing will give us an error to let us know that the data was invalid. We can also chain refine functions to add checks for our password complexity rules: ` Here we've chained multiple refine functions. You could alternatively use superRefine, which gives you even more fine grained control. Now that we've built out our schema and added refinements for extra validation, we can parse some user inputs. Let's see two test cases: one that's bound to fail, and one that will succeed. ` There are two main ways we can use our schema to validate our data: parse and safeParse. The main difference is that parse will throw an error if validation fails, while safeParse will return an object with a success property of either true or false, and either a data property with your parsed data or an error property with the details of a ZodError explaining why the parsing failed. In the case of our example data, userInput2 will parse just fine and return the data for you to use. But userInput1 will create a ZodError listing all of the ways it has failed validation. ` ` We can use these error messages to communicate to the user how they need to fix their form inputs if validation fails. Each error in the list describes the validation failure and gives us a human readable message to go with it. You'll notice that the validation errors for checking for a valid email and for checking password length have a lot of details, but we've got three items at the end of the error list that don't really tell us anything useful: just a custom error of Invalid input. The first is from our refine checking if the passwords match, and the second two are from our refine functions checking for password complexity (numbers and special characters). Let's modify our refine functions so that these errors are useful! We'll add our own error parameters to customize the message we get back and the path to the data that failed validation. ` Now, our error messages from failures in refine are informative! You can figure out which form fields aren't validating from the path, and then display the messages next to form fields to let the user know how to remedy the error. ` By giving our refine checks a custom path and message, we can make better use of the returned errors. In this case, we can highlight specific problem form fields for the user and give them the message about what is wrong. Integrating with TypeScript Integrating Zod with TypeScript is very easy. Using z.infer<typeof YourSchema> will allow you to avoid writing extra TypeScript types that merely reflect the intent of your Zod schemas. You can create a type from any Zod schema like so: ` Using a TypeScript type derived from a Zod schema does *not* give you any extra level of data validation at the type level beyond what TypeScript is capable of. If you create a type from z.string.min(3).max(20), the TypeScript type will still just be string. And when compiled to JavaScript, even that will be gone! That's why you still need to use parse/safeParse on incoming data to validate it before proceeding as if it really does match your requirements. A common pattern with inferring types from Zod schemas is to use the same name for both. Because the schema is a variable, there's no name conflict if the type uses the same name. However, I find that this can lead to confusing situations when trying to import one or the other—my personal preference is to name the Zod schema with Schema at the end to make it clear which is which. Conclusion Zod is an excellent tool for easily and confidently asserting that the data you're working with is exactly the sort of data you were expecting. It gives us the ability to assert at runtime that we've got what we wanted, and allows us to then craft strategies to handle what happens if that data is wrong. Combined with the ability to infer TypeScript types from Zod schemas, it lets us write and run more reliable code with greater confidence....

Nov 15, 2024

7 mins

JavaScriptZod

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Integrating Next.js with New Relic

Integrating Next.js with New Relic

Common Prerequisite Steps

Self-Hosting Next.js

Optionally, Logging via New Relic API

Conclusion

Dario Djuric

You might also like

Tim Neutkens, Co-Author of Next.js on the State of Next

Maximizing Server Rendering for Interactive Next.js Applications

A Look at Playwright Parallelism

Introduction to Zod for Data Validation

Let's innovate together!

You might also like

Tim Neutkens, Co-Author of Next.js on the State of Next

Maximizing Server Rendering for Interactive Next.js Applications

A Look at Playwright Parallelism

Introduction to Zod for Data Validation