Javascript

Integrating AI Models Locally with Next.js ft. Jesus Padron

Published Sep 3, 2024

Updated Dec 11, 2024

1 min read

Jesus Padron from the This Dot team shows you how to integrate AI models into a Next.js application. Jesus walks through the process of running Meta's Llama 3.1 model locally, leveraging OpenAI's Whisper for speech-to-text conversion, and using OpenAI's TTS model for text-to-speech conversion. By the end of the episode, listeners will know how to create an AI voice assistant that processes voice input, understands the content, and responds audibly.

Chapters:

Introduction to the Episode (00:00:03)
Overview of Llama 3.1 and Setup (00:02:14)
Setting Up the Next.js Application (00:04:40)
Recording Audio with MediaRecorder API (00:11:37)
Integrating OpenAI's Whisper for Speech-to-Text (00:36:46)
Generating Responses with Llama 3.1 (00:48:24)
Implementing Text-to-Speech with OpenAI's TTS (01:03:26)
Final Testing and Demonstration (01:06:37)
Summary and Next Steps (01:09:01)
Closing Remarks (01:14:19)

Follow Jesus on Social Media Twitter: https://x.com/padron4497 Github: https://github.com/padron4497

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Improving INP in React and Next.js

Improving INP in React and Next.js In one of my previous articles, I've explained what INP is, how it works, and how it may affect your website. I also promised you to follow up with more concrete advice on how to improve your INP in your favorite framework. This is the follow-up article, where I'll focus on how to improve your INP score in React and Next.js. How to prepare for INP in React and Next.js? The first thing to do is to ensure you're using the latest version of React. The React team has been working on making React more INP-friendly and has already made some improvements in the latest versions. To enhance your INP score, consider fully taking advantage of new features introduced in React 18, such as Concurrent Rendering, Automatic Batching, and Selective Hydration. However, there are also some general areas to focus on, such as SSR and SSG in Next.js, Web Workers, or optimizing your hooks and state management. Concurrent Rendering The Concurrent Mode in React uses an algorithm that breaks rendering down into so-called "fiber nodes" and schedules the renders based on their expiration and priority. This effectively allows the user to interact with the page while the rendering is still in progress. In previous React versions, all updates, such as setState calls were treated as "urgent" and once the re-render had started, there was no way to interrupt it. Concurrent Mode changes this by being able to prioritize the updates and interrupt a non-blocking state update started with startTransition. For a simple explanation of concurrency in React, you can check out Dan Abramov's explanation. As part of the Concurrent Mode, React introduced several lifecycle methods that allow you to prioritize the rendering of certain parts of your UI, such as: - useTransition hook that allows you to update the state without blocking the UI, - useDeferredValue hook that allows you to defer the rendering of certain parts of your UI, - startTransition API that, similarly to the useTransition hook lets you mark a state update as non-blocking. It lacks, however, an indication of whether it's still pending. Automatic Batching Introduced in React 18, Automatic Batching reduces the number of re-renders that happen on state changes even when they happen outside of event handlers, e.g. in a setTimeout or Promise callback. This feature comes out of the box and you don't have to do anything to enable it, and it makes a great argument for upgrading to React 18. Selective Hydration Selective Hydration allows you to take hydration off the main thread by wrapping your components in a Suspense boundary. This way, components can become interactive faster as the browser can do other work on the main thread while the hydration is happening. To fully take advantage of selective hydration, consider the following: - Prioritizing Above-the-Fold Content: Use Suspense boundaries strategically around any parts of your application that may take the server longer to deliver to ensure they don’t block critical content from becoming interactive as soon as possible. - Hydration on Interaction: Implementing hydration upon user interaction for non-critical components can drastically reduce the main thread's workload, enhancing INP. Vercel even has a small case study showing how using selective hydration improved the performance of a Next.js site. Server-Side Rendering (SSR) and Static Site Generation (SSG) in Next.js Not everything has to run client-side. Next.js excels in SSR and SSG capabilities, which can significantly impact INP by delivering content to users faster. Optimizing SSR with techniques like incremental static regeneration (ISR) or leveraging SSG for static pages ensures that users can interact with content faster, improving the perceived performance. Workers Offloading heavy computations to Web Workers can free up the main thread, enhancing the responsiveness of React and Next.js applications. This strategy is especially useful when dealing with third-party scripts. Offloading such scripts in Next.js can be easily done by specifying the "worker" strategy on your Script component. Be aware that this feature is not yet stable and does not work with the app directory, though. If you want to take things one step further, you could use Partytown, which helps you offload any resource-intensive scripts to Web Workers. It comes with a React component that you can use to wrap your third-party scripts and offload them to a Web Worker, and it's compatible with Next.js as well. Hooks and State Management State management in React applications can easily get out of hand, leading to unnecessary re-renders and effectively an increased INP. Sometimes, using a state management library like Redux or MobX can help you consolidate your state and reduce the number of re-renders. However, they are not silver bullets and can also introduce performance issues if not used properly. If you are dealing with a lot of re-renders due to prop changes, make sure you are leveraging memoization. As of now, you may need to work with useMemo and useCallback hooks to memoize your values and functions, respectively. The upcoming React 19’s Forget Compiler, however, will apparently memoize everything under the hood, making these hooks obsolete. Using memoization properly can help you reduce the number of re-renders and improve your INP. To investigate your hook dependencies and re-renders, you can leverage React Developer Tools or use this handy helper hook I found on the internet to trace your re-renders: ` Conclusion Improving INP in React and Next.js is not easy and can require much investigation and fine-tuning. Still, it's worth doing to avoid being penalized by Google in its search results and provide a better experience for your users. Adopting React 18's new features, leveraging SSR and SSG in Next.js, utilizing Web Workers, and optimizing hooks and state management can significantly boost your INP score and deliver a faster application to your users. Remember, INP is just one among many performance metrics emphasizing the need for a comprehensive approach to performance optimization...

Apr 19, 2024

5 mins

Web PerformanceReactNextJSJavaScript

Communication Between Client Components in Next.js

Communication Between Client Components in Next.js In recent years, Next.js has become one of the most popular React frameworks for building server-rendered applications. With the introduction of the App Router in Next.js 13, the framework has taken a major leap forward by embracing a new approach to building web applications: the concept of server components and client components. This separation of concerns allows developers to strategically decide which parts of their application should be rendered on the server and which are then hydrated in the browser for interactivity. The Challenge: Communicating Between Client Components While server components offer numerous benefits, they also introduce a new challenge: how can client components within different boundaries communicate with each other? For instance, let's consider a scenario where you have a button (a client component) and a separate client component that displays the number of times the button has been clicked. In a regular React application, this would typically be accomplished by lifting the state to a common ancestor component, allowing the button to update the state, which is then passed down to the counter display component. However, the traditional approach may not be as straightforward in a Next.js application that heavily relies on server components, with client components scattered across the page. This blog post will explore three ways to facilitate communication between client components in Next.js, depending on where you want to store the state. Lifting State to a Common Client Component One approach to communicating between client components is to lift the state to a common client component. This could be a React context provider, a state management system like Zustand, or any other solution that allows you to share state across components. The key aspect is that this wrapper component should be higher up in the tree (perhaps even in the layout) and accept server components as children. Next.js allows you to interleave client and server components as much as you want, as long as server components are passed to client components as props or as component children. Here's how this approach might look in practice. First of all, we'd create a wrapper client component that holds the state: ` This wrapper component can be included in the layout: ` Any server components can be rendered within the layout, potentially nesting them several levels deep. Finally, we'd create two client components, one for the button and one for the counter display: ` ` The entire client and server component tree is rendered on the server, and the client components are then hydrated in the browser and initialized. From then on, the communication between the client components works just like in any regular React application. Check out the page using this pattern in the embedded Stackblitz window below: Using Query Params for State Management Another approach is to use query params instead of a wrapper client component and store the state in the URL. In this scenario, you have two client components: the button and the counter display. The counter value (the state) is stored in a query param, such as counterValue. The client components can read the current counter value using the useSearchParams hook. Once read, the useRouter hook can update the query param, effectively updating the counter value. However, there's one gotcha to this approach. If a route is statically rendered, calling useSearchParams will cause the client component tree up to the closest Suspense boundary to be client-side rendered. Next.js recommends wrapping the client component that uses useSearchParams in a boundary. Here's an example of how this approach might look. The button reads the current counter value and updates it on click by using the router's replace function: ` The counter display component is relatively simple, only reading the counter value: ` And here is the page that is a server component, hosting both of the above client components: ` Feel free to check out the above page in the embedded Stackblitz below: Storing State on the Server The third approach is to store the state on the server. In this case, the counter display component accepts the counter value as a prop, where the counter value is passed by a parent server component that reads the counter value from the database. The button component, when clicked, calls a server action that updates the counter value and calls revalidatePath() so that the counter value is refreshed, and consequently, the counter display component is re-rendered. It's worth noting that in this approach, unless you need some interactivity in the counter display component, it doesn't need to be a client component – it can be purely server-rendered. However, if both components need to be client components, here's an example of how this approach might look. First, we'll implement a server action that updates the counter value. We won't get into the mechanics of updating it, which in a real app would require a call to the database or an external API - so we're only commenting that part. After that, we revalidate the path so that the Next.js caches are purged, and the counter value is retrieved again in server components that read it. ` The button is a simple client component that calls the above server action when clicked. ` The counter display component reads the counter value from the parent: ` While the parent is a server component that reads the counter value from the database or an external API. ` This pattern can be seen in the embedded Stackblitz below: Conclusion Next.js is a powerful framework that offers various choices for implementing communication patterns between client components. Whether you lift state to a common client component, use query params for state management, or store the state on the server, Next.js provides all the tools you need to add a communication path between two separate client component boundaries. We hope this blog post has been useful in demonstrating how to facilitate communication between client components in Next.js. Check out other Next.js blog posts we've written for more insights and best practices. You can also view the entire codebase for the above snippets on StackBlitz....

Jun 21, 2024

5 mins

NextJSReact

Svelte 5 is Here!

Svelte 5 was finally released after a long time in development. Fortunately, we've been able to test it for some time, and now it has a stable release. Let's dig into its features and why this is such a significant change, even though Svelte 4 code is almost 100% compatible. Svelte syntax everywhere This is one of my favorite additions. Previously, Svelte syntax was limited to a component element. Refactoring code from a component and moving it to a JavaScript file worked differently. Now you can use the .svelte.js or .svelte.ts extension, which allows you to use the same syntax everywhere. It's important to note that it's a way to express that this is not just JS, and what you write may be compiled to something else, just like .svelte files are not just html even though they look very similar. Runes The introduction of runes is one of the most significant changes in Svelte. Many users felt attracted to variables being instantly reactive in previous versions. ` There was, however, a lot of magic underneath and a considerable amount of work from the compiler to make it behave reactively. $state In Svelte 5, a reactive variable has to be explicitly declared using the $state rune, which brings a clear distinction between what is reactive and what isn’t. ` In the previous code, $state is not an actual function being called; it's a hint for the compiler to do something special with this declaration. Rune names always start with a dollar sign ($) and do not need to be imported. However, there’s much more to the changes than just the way we declare reactive variables. Runes bring a lot of new features that make Svelte 5 a great improvement. In Svelte 5, objects or arrays use Proxies to allow for granular reactivity, meaning that individual properties in an object are reactive, even if they are nested objects or arrays. If you modify a property, it will only update that property and will not trigger an update on the whole object. It also supports triggering updates when methods like array.push are called. In previous versions, an assignment was required to trigger an update: ` To have a similar behavior of svelte 4 (no deep reactivity), use the $state.raw() rune. This syntax of .* is common for related features of a rune: $state.raw() will require an assignment to trigger reactivity. ` Because proxies are used, you may need to extract the underlying state instead of the proxy itself. In that case, $state.snapshot() should be used: ` $derived We will use $derived and $derived.by to declare a derived state. The two runes are essentially the same, except one allows us to use a function instead of an expression, allowing for more complex operations to calculate the derived values. ` $effect Effects are useful for running something triggered by a change in state. One of the things you'll note from the $effect rune is that dependencies don't need to be explicit. Those will be picked from reactive values read synchronously in the body of the effect function. ` Something to bear in mind is that these dependencies are picked each time the effect runs. The values read during the last run become the dependencies of the effect. ` Depending on the result of the random method, foo or bar will stop being a dependency of the effect. You should place them outside the condition so they can trigger reruns of the effect. *Variants* of the effect rune are $effect.pre, which runs before a DOM update, and $effect.tracking(), which checks for the context of the effect (true for an effect inside an effect or in the template). $props This rune replaces the export let keywords used in previous versions to define a component's props. To use the $props syntax, we can consider it to return an object with all the properties a component receives. We can use JavaScript syntax to destructure it or use the rest property if we want to retrieve every property not explicitly destructured.. ` $bindable If you want to mutate a prop so that the change flows back to the parent, you can use the $bindable prop to make it work in both directions. A parent component can pass a value to a child component, and the child component is able to modify it, returning the data back to the parent component. ` $inspect The $inspect rune only works in dev mode and will track dependencies deeply. By default, it will call console.log with the provided argument whenever it detects a change. To change the underlying function, use $inspect.with().with): ` $host The host rune gives access to the host element when compiling as a custom element: ` ` Other changes Another important change is that component events are just props in Svelte 5, so you can destructure them using the $props() rune: ` A special children property can be used to project content into a component instead of using slots. Inside the components, use the [@render[(https://svelte.dev/docs/svelte/@render) tag to place them. ` Optional chaining (?.) prevents from attempting to render it if no children are passed in. If you need to render different components in different places (named slots before Svelte 5, you can pass them as any other prop, and use @render with them. Snippets Snippets allow us to declare markup slices and render them conveniently using the render tag (@render). They can take any number of arguments. ` Snippets can also be passed as props to other components. ` ` Conclusion Some exciting changes to the Svelte syntax were introduced while, at the same time, maximum compatibility efforts were made. This was a massive rewrite with numerous improvements in terms of performance, bundle size, and DX. Besides that, a new CLI has been released, making the whole experience of starting a project or adding features delightful. If you haven't tried Svelte before, it's a great time to try it now....

Nov 27, 2024

4 mins

Svelte

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Understanding the Difference Between :focus and :focus-visible in CSS I have learned my fair share about the importance of keyboard accessibility, so I know that visual indication of the focused element is very important. But the well-known :focus pseudo-class is not always the best fit for this job. That's where :focus-visible comes in. Let's look at the differences between these two pseudo-classes and explore the best practices for using them effectively. What is the :focus Pseudo-Class? The :focus pseudo-class is a CSS selector that applies styles to any element that receives focus, regardless of how that focus was triggered. This includes focus events from keyboard navigation, mouse clicks, and touch interactions. Example Usage of :focus ` In this example, the button will display a blue outline whenever it is focused, whether the user clicks on it with a mouse, taps it on a touchscreen, or navigates to it using the keyboard. What is the :focus-visible Pseudo-Class? The :focus-visible pseudo-class is more specialized. It only applies styles to an element when the browser determines that the focus should be visible. This typically occurs when the user navigates via the keyboard or assistive technologies rather than through mouse or touch input. Example Usage of :focus-visible ` Here, the button will only show a blue outline when focused through keyboard navigation or another input method that usually requires visible focus indicators. Key Differences Between :focus and :focus-visible :focus - Behavior: Applies to any element that receives focus, regardless of the input method. - Use Cases: Ensures that all interactions with the element are visually indicated, whether by mouse, keyboard, or touch. :focus-visible - Behavior: Applies styles only when the focus should be visible, such as using a keyboard or assistive technology. - Use Cases: Ideal for scenarios where you want to provide focus indicators only to keyboard and assistive technology users while avoiding unnecessary outlines for mouse and touch users, typically required by design. Accessibility Implications :focus - Pros: - Guarantees that all users can see when an element is focused, which is critical for accessibility. - Cons: - Can lead to a suboptimal experience for mouse users, as focus styles may appear unnecessarily during mouse interactions. :focus-visible - Pros: - Enhances user experience by showing focus indicators only when necessary, thus keeping the interface clean for mouse and touch users. - Tailors the experience for keyboard and assistive technology users, providing them with clear visual cues. - Cons: - Additional considerations may be required to ensure that focus indicators are not accidentally omitted, especially in older browsers that do not support :focus-visible. - There may be cases where you want to show focus indicators for all users, regardless of input method. Best Practices for Using :focus and :focus-visible To achieve the best accessibility and user experience, combining both :focus and :focus-visible in your CSS is often a good idea. Combining :focus and :focus-visible ` Here is a Stackblitz example of what such styling could look like for you to try out and play with. Additional Tips - Test with Keyboard and Assistive Technology: Ensure that your web application is navigable using a keyboard (Tab, Shift + Tab, etc.) and that focus indicators are visible for those who rely on them. It's never a bad idea to include accessibility testing in your e2e testing suite. - Provide Clear Focus Indicators: Make sure that focus indicators are prominent and easy to see. A subtle or hard-to-spot focus indicator can severely impact accessibility for users who rely on keyboard navigation. Conclusion The :focus-visible pseudo-class offers a more refined way to manage focus indicators, improving accessibility and user experience, particularly for keyboard and assistive technology users. By understanding the differences between :focus and :focus-visible, and applying best practices in your CSS, you can create more accessible and user-friendly web applications. Remember, accessibility should never be an afterthought. By thoughtfully applying focus styles, you ensure that all users, regardless of how they interact with your site, can easily navigate and interact....

Nov 8, 2024

3 mins

AccessibilityCSSPlaywright

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Integrating AI Models Locally with Next.js ft. Jesus Padron

You might also like

Improving INP in React and Next.js

Communication Between Client Components in Next.js

Svelte 5 is Here!

Understanding the Difference Between `:focus` and `:focus-visible` in CSS

Let's innovate together!

You might also like

Improving INP in React and Next.js

Communication Between Client Components in Next.js

Svelte 5 is Here!

Understanding the Difference Between `:focus` and `:focus-visible` in CSS