ReactJS

Getting started with 3D and React

Published Nov 15, 2021

Updated Feb 13, 2023

3 min read

This article was written over 18 months ago and may contain information that is out of date. Some content may be relevant but please refer to the relevant official documentation or available resources for the latest information.

Introduction

In this tutorial, we will build 3D designs with React using the react-three-fiber package. What is react-three-fiber? It’s a three.js for React.js, and it is a powerful renderer that helps render 3D models and animations for React, and its native applications.

Install

First, let’s create a new React project or Next.JS project.

npx create-react-app react-3d

Then, we need to install two packages:three and react-three-fiber.

npm: npm install three @react-three/fiber

yarn: yarn add three @react-three/fiber

Create your first sense

Go to App.js and copy and paste the following code:

import React, { useRef, useState } from 'react';
import { Canvas, useFrame } from '@react-three/fiber';

function Box(props) {
  // This reference will give us direct access to the mesh
  const mesh = useRef();
  // Set up state for the hovered and active state
  const [hovered, setHover] = useState(false);
  const [active, setActive] = useState(false);
  // Subscribe this component to the render-loop, rotate the mesh every frame
  useFrame((state, delta) => (mesh.current.rotation.x += 0.01));
  // Return view, these are regular three.js elements expressed in JSX
  return (
    <mesh
      {...props}
      ref={mesh}
      scale={active ? 1.5 : 1}
      onClick={(event) => setActive(!active)}
      onPointerOver={(event) => setHover(true)}
      onPointerOut={(event) => setHover(false)}
    >
      <boxGeometry args={[1, 1, 1]} />
      <meshStandardMaterial color={hovered ? 'hotpink' : 'orange'} />
    </mesh>
  );
}

function App() {
  return (
    <Canvas style={{ height: '100vh' }}>
      <ambientLight />
      <pointLight position={[10, 10, 10]} />
      <Box position={[-1.2, 0, 0]} />
      <Box position={[1.2, 0, 0]} />
    </Canvas>
  );
}

export default App;

Save. The result will look like this:

So, let’s look at the code:

Canvas is your main scene.
mesh is the box objects.
pointLight is the light for the scene.
ambientLight is a global light that illuminates all objects in the scene equally, and this light cannot be used to cast shadows as it does not have a direction.

Cameras

We have three types of cameras that react-fiber-three support:

PerspectiveCamera: that can set itself as the default.
OrthographicCamera: with this camera, all objects appear at the same scale.
CubeCamera: Creates 6 cameras that render the scene

Events

React three fiber package offers us a lot of events that you can use it make your users interact with the 3D design like we saw with the Box element in the first example.

Some of the events:

onClick: When the user clicks on the object.
onContextMenu: When the user opens the context menu on the object (Right click)
onDoubleClick: When the user double clicks on the object.
onWheel: When the user uses the mouse wheel to scroll on the object.
onPointerUp: When the user moves the cursor up.
onPointerDown: When the user moves the cursor down.
onPointerOver: When the user hovers with the cursor on the object.
onPointerOut: When the user moves out of the object with the cursor.
onUpdate: When the object is being updated.

Loading models

With react-three-fiber and three, you can import 3D models and use them in your scene! There are many types of 3D model extensions that you can import with this method, including GLTF, FBX, OBJ, etc...

To import an external model, you will first need to import two things in your file: useLoader hook from react-three-fiber, and the model extension loader from three.js.

For example:

import { useLoader } from '@react-three/fiber';
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader';

And then import the model using the useLoader hook.

const gltf = useLoader(GLTFLoader, '/Model.gltf');

Finally, you can use it in your scene.

<primitive object="{gltf.scene}" />

You can see a live example here.

Animation

Finally, we will talk about how to animate your models. We will need to import useFrame hook! It is a Fiber hook that lets you execute code on every frame of Fiber's render loop. This can have a lot of uses, but we will focus on building an animation with it.

First, import it.

import { useFrame } from '@react-three/fiber';

Create a react ref:

const myMesh = React.useRef();

Then connect this ref with your mesh.

<mesh ref="{myMesh}"></mesh>

Finally, start animating it using useFrame.

useFrame(({ clock }) => {
  myMesh.current.rotation.x = clock.getElapsedTime();
  myMesh.current.rotation.y = clock.getElapsedTime();
});

In this article, we walked through how to install react-three-fiber, and how to make basic objects. In the next article, we will learn how to create a product landing page with it, so stay tuned!

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

Mark Shenouda
@MarkSShenouda @markshenouda

Making Seamless Page Transitions with the View Transitions API

Make Seamless Page Transitions using the View Transitions API Traditionally web applications have always had a less polished experience, both functionally and visually speaking, compared to native applications on mobile and other platforms. This has been improving over the years with the introduction of new web APIs and standards. One such new API that bridges that gap is the View Transitions API. What is the View Transitions API? The View Transitions API is a new browser feature that allows developers to more easily create animated transitions between pages and views, similar to how mobile apps have transitions when navigating between pages. Adding view transitions to your application can reduce the cognitive load on your users and make the experience feel less inconsistent. One great thing about view transitions is that they can be used by both SPAs (single-page applications) and MPAs (multi-page applications) alike! Using the View Transitions API for SPAs Let’s see how we can use view transitions ourselves. We will use examples adapted from demos created by the wonderful Jake Archibald and Bramus. There are some alterations to a few of the examples to make them work with the newest version of the View Transitions API, but otherwise, they’re mostly the same. The process of getting view transitions working for SPAs is as simple as calling document.startViewTransition right before starting navigation and replacing your page’s content immediately after that. You need to make this call in your router or in an event listener that triggers your page navigation. The following is an example of how you can hook into page navigation in a vanilla JavaScript application. This example uses the Navigation API if it’s available. As of the time of writing this article only Chromium-based browsers support the Navigation API, but there is a fallback to using a click listener on link elements so that this works with Firefox and Safari. ` If you’re using a framework, then there’s probably a better way to do this specific to that framework than by using this contrived example. For example, if you are implementing usage of this API with a Next application, you may opt for something like next-view-transitions. For this article, we’re going to focus on the way this is done in vanilla JavaScript so that the fundamentals of using the API are understood. Knowing how it’s implemented in vanilla JavaScript should give you enough understanding to implement this in the framework of your choice if there isn’t already a library that does it for you. Now we have a way to know when a navigation is being requested. Using the above utility function we can start the animation and replace our content as follows: ` getMyPageContentSomehow can be any function that returns HTML. How that is implemented will depend on your application and framework of choice. If you want to run these examples on your machine, the demo repository has instructions in its README. In essence, it’s as simple as hosting an HTTP server at the root of the project and visiting localhost. No fancy build system is required! By default, a fade is done that only lasts for a moment. The animation that happens can be configured using CSS with the ::view-transition-group pseudo-element on the element you want to animate, which in our case is the entire page, and we pass in root. ` Given that the animation properties are used, you have much control over what kind of animation to do. You can change the duration and the interpolation, add keyframes and gradients, and even add image masks. You can learn more about the animation properties in CSS on MDN. Using the View Transitions API for MPAs As mentioned, view transitions also work for MPAs with no client-side routing. Setting up view transitions for MPAs is much easier than for SPAs. All you have to do is include the CSS rule on all pages where you want the animations. ` Of course, like how it works with SPAs, you can customize the animations to your heart’s content using the view transition pseudo-elements. Conclusion I hope this article helped you understand the fundamentals of using the View Transitions API. Now that most browsers support it and it’s not too difficult to add it to existing applications, it is a great time to learn how to use it. You can view a full list of examples here (made by Jake Archibald and Bramus) if you want to see how advanced the animations can get. I like the Star Wars ones especially. The README explains how to get all the examples up and running (a total of 43!)....

Aug 2, 2024

4 mins

JavaScript

How to build an AI assistant with OpenAI, Vercel AI SDK, and Ollama with Next.js

In today’s blog post, we’ll build an AI Assistant using three different AI models: Whisper and TTS from OpenAI and Llama 3.1 from Meta. While exploring AI, I wanted to try different things and create an AI assistant that works by voice. This curiosity led me to combine OpenAI’s Whisper and TTS models with Meta’s Llama 3.1 to build a voice-activated assistant. Here’s how these models will work together: * First, we’ll send our audio to the Whisper model, which will convert it from speech to text. * Next, we’ll pass that text to the Llama 3.1 model. Llama will understand the text and generate a response. * Finally, we’ll take Llama’s response and send it to the TTS model, turning the text back into speech. We’ll then stream that audio back to the client. Let’s dive in and start building this excellent AI Assistant! Getting started We will use different tools to build our assistant. To build our client side, we will use Next.js. However, you could choose whichever framework you prefer. To use our OpenAI models, we will use their TypeScript / JavaScript SDK. To use this API, we require the following environmental variable: OPENAI_API_KEY— To get this key, we need to log in to the OpenAI dashboard and find the API keys section. Here, we can generate a new key. Awesome. Now, to use our Llama 3.1 model, we will use Ollama and the Vercel AI SDK, utilizing a provider called ollama-ai-provider. Ollama will allow us to download our preferred model (we could even use a different one, like Phi) and run it locally. The Vercel SDK will facilitate its use in our Next.js project. To use Ollama, we just need to download it and choose our preferred model. For this blog post, we are going to select Llama 3.1. After installing Ollama, we can verify if it is working by opening our terminal and writing the following command: Notice that I wrote “llama3.1” because that’s my chosen model, but you should use the one you downloaded. Kicking things off It's time to kick things off by setting up our Next.js app. Let's start with this command: ` After running the command, you’ll see a few prompts to set the app's details. Let's go step by step: * Name your app. * Enable app router. The other steps are optional and entirely up to you. In my case, I also chose to use TypeScript and Tailwind CSS. Now that’s done, let’s go into our project and install the dependencies that we need to run our models: ` Building our client logic Now, our goal is to record our voice, send it to the backend, and then receive a voice response from it. To record our audio, we need to use client-side functions, which means we need to use client components. In our case, we don’t want to transform our whole page to use client capabilities and have the whole tree in the client bundle; instead, we would prefer to use Server components and import our client components to progressively enhance our application. So, let’s create a separate component that will handle the client-side logic. Inside our app folder, let's create a components folder, and here, we will be creating our component: ` Let’s go ahead and initialize our component. I went ahead and added a button with some styles in it: ` And then import it into our Page Server component: ` Now, if we run our app, we should see the following: Awesome! Now, our button doesn’t do anything, but our goal is to record our audio and send it to someplace; for that, let us create a hook that will contain our logic: ` We will use two APIs to record our voice: navigator and MediaRecorder. The navigator API will give us information about the user’s media devices like the user media audio, and the MediaRecorder will help us record the audio from it. This is how they’re going to play out together: ` Let’s explain this code step by step. First, we create two new states. The first one is for keeping track of when we are recording, and the second one stores the instance of our MediaRecorder. ` Then, we’ll create our first method, startRecording. Here, we are going to have the logic to start recording our audio. We first check if the user has media devices available thanks to the navigator API that gives us information about the browser environment of our user: If we don’t have media devices to record our audio, we just return. If they do, then let us create a stream using their audio media device. ` Finally, we go ahead and create an instance of a MediaRecorder to record this audio: ` Then we need a method to stop our recording, which will be our stopRecording. Here, we will just stop our recording in case a media recorder exists. ` We are recording our audio, but we are not storing it anywhere. Let’s add a new useEffect and ref to accomplish this. We would need a new ref, and this is where our chunks of audio data will be stored. ` In our useEffect we are going to do two main things: store those chunks in our ref, and when it stops, we are going to create a new Blob of type audio/mp3: ` It is time to wire this hook with our AudioRecorder component: ` Let’s go to the other side of the coin, the backend! Setting up our Server side We want to use our models on the server to keep things safe and run faster. Let’s create a new route and add a handler for it using route handlers from Next.js. In our App folder, let’s make an “Api” folder with the following route in it: We want to use our models on the server to keep things safe and run faster. Let’s create a new route and add a handler for it using route handlers from Next.js. In our App folder, let’s make an “Api” folder with the following route in it: ` Our route is called ‘chat’. In the route.ts file, we’ll set up our handler. Let’s start by setting up our OpenAI SDK. ` In this route, we’ll send the audio from the front end as a base64 string. Then, we’ll receive it and turn it into a Buffer object. ` It’s time to use our first model. We want to turn this audio into text and use OpenAI’s Whisper Speech-To-Text model. Whisper needs an audio file to create the text. Since we have a Buffer instead of a file, we’ll use their ‘toFile’ method to convert our audio Buffer into an audio file like this: ` Notice that we specified “mp3”. This is one of the many extensions that the Whisper model can use. You can see the full list of supported extensions here: https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-file Now that our file is ready, let’s pass it to Whisper! Using our OpenAI instance, this is how we will invoke our model: ` That’s it! Now, we can move on to the next step: using Llama 3.1 to interpret this text and give us an answer. We’ll use two methods for this. First, we’ll use ‘ollama’ from the ‘ollama-ai-provider’ package, which lets us use this model with our locally running Ollama. Then, we’ll use ‘generateText’ from the Vercel AI SDK to generate the text. Side note: To make our Ollama run locally, we need to write the following command in the terminal: ` ` Finally, we have our last model: TTS from OpenAI. We want to reply to our user with audio, so this model will be really helpful. It will turn our text into speech: ` The TTS model will turn our response into an audio file. We want to stream this audio back to the user like this: ` And that’s all the whole backend code! Now, back to the frontend to finish wiring everything up. Putting It All Together In our useRecordVoice.tsx hook, let's create a new method that will call our API endpoint. This method will also take the response back and play to the user the audio that we are streaming from the backend. ` Great! Now that we’re getting our streamed response, we need to handle it and play the audio back to the user. We’ll use the AudioContext API for this. This API allows us to store the audio, decode it and play it to the user once it’s ready: ` And that's it! Now the user should hear the audio response on their device. To wrap things up, let's make our app a bit nicer by adding a little loading indicator: ` Conclusion In this blog post, we saw how combining multiple AI models can help us achieve our goals. We learned to run AI models like Llama 3.1 locally and use them in our Next.js app. We also discovered how to send audio to these models and stream back a response, playing the audio back to the user. This is just one of many ways you can use AI—the possibilities are endless. AI models are amazing tools that let us create things that were once hard to achieve with such quality. Thanks for reading; now, it’s your turn to build something amazing with AI! You can find the complete demo on GitHub: AI Assistant with Whisper TTS and Ollama using Next.js...

Sep 27, 2024

8 mins

AINextJSJavaScript

Build Beautiful Storefronts Quickly with Shopify and Next

Introduction Shopify is one of the world’s best e-commerce platforms for building online stores of any size. It’s a hosted platform, which means you don’t have to worry about the technical details of managing a server or dealing with software updates. You can focus on building your business instead. Next.js is a React framework for building static and server-side rendered applications. It’s a great choice for building e-commerce storefronts and enables you to do more customization, and it’s what we’ll be using in this article. Shopify Next.js Starter Kit Recently, we’ve created a starter kit that you can use to build your own Shopify storefront with Next.js. It’s a great way to get started quickly especially since it is powered by the new App Router and React Server Components, and it’s completely free. You can find it on GitHub here: Shopify Next.js Starter Kit We also have a live demo of the starter kit here: Shopify Next.js Starter Kit Demo Getting Started To get started, open your terminal and run the following command: And choose Shopify, NextJS 13.4 and Tailwind CSS Then, choose the project name And everything ready to go, next steps are to go to the directory and install the packages Setup Shopify Account Next, you’ll need to create a Shopify store. There are two ways to get a Shopify account: 1- You can do this by going to Shopify and clicking on the “Start free trial” button and create a Shopify account. 2- Or you can create a Shopify Partner Account for development purposes Once you’ve created your store, you’ll need to create a private app. You can do this by going to the “Apps” section of your Shopify admin and clicking on the “Manage private apps” link. Then click on the “Create a new private app” button. Give your app a name, and then click on the “Save” button. You’ll then be taken to a page where you can see your API credentials. You’ll need to copy these credentials into your .env.local file. You can find the .env.local file at the root of your project. It should look something like this: Modify the design After adding the required secrets, run npm run dev to run the development server locally The project structure is simple and easy. Since we are using the App Router, all of the routes are under /app folder and the shared components are under /components folder. This structure make it easy for you to edit and modify easily on the design Also all of the components have been written in a clean way with Tailwind CSS to make it easy for you to edit it. Deploy Since it’s a Next.js project, its deployment is easier than ever, most of the modern host providers support deploying it with just one click like * Vercel * Netlify * Cloudflare Page * AWS Amplify * Render * Fly.io Just push the project to a remote git repository using GitHub and connect it to the hosting provider of your choice. Conclusion In this article, we’ve shown you how to build a Shopify storefront with Next.js with our new starter kit from Starter.dev. We’ve also shown you how to use the new App Router and React Server Components to build a fast and performant storefront. We hope you’ve enjoyed this article and found it useful. If you have any questions or comments, please feel free to reach out to us on Twitter or GitHub. We’d love to hear from you!...

Nov 1, 2023

4 mins

NextJSShopify

Building a Stripe App: A Step-by-Step Guide to QR Code Generation

Building a Stripe App: A Step-by-Step Guide to QR Code Generation Why Build a Stripe App? I recently participated in an audio space with the Stripe team, and something they said really stuck with me: the Stripe app store is a growing area that isn't overly saturated yet. There's a lot of potential for new apps, and companies can use this opportunity to grow. I work at a company called This Dot Labs, and we've created several Stripe apps and even own one. After looking at the data, I can confirm that the Stripe team was right! Creating a QR Code Generator App For this tutorial, we'll build a QR code app that can take a URL and generate a code for it. This is a good use case to help you understand the ins and outs of Stripe's developer tools. Why QR Codes? QR codes are useful tools that have become common in e-commerce, restaurants, and other industries. While Stripe already has a QR code tool, we'll make our own to familiarize ourselves with their syntax and problem-solving approaches. Project Structure Before we dive into the implementation, let's look at the structure of our Stripe App QR Code project: * .vscode: Contains settings for Visual Studio Code * source/views: Holds the main application views * .gitignore: Specifies files to ignore in version control * stripe-app.json: Defines the Stripe app configuration * ui-extensions.d.ts: TypeScript declaration file for UI extensions * .build: this is where the built Stripe app gets placed. Step-by-Step Implementation 1\. Install Stripe Locally First, you need to install Stripe on your local machine. The documentation provides great instructions for this: * For Mac users: Use Brew to install * For Windows users: Download the package and add it to your environment variables You can find the details here in the stripe docs to install the Stripe CLI https://docs.stripe.com/stripe-cli When using Windows, you must do stripe login from Powershell, NOT from Git bash or any other tool. After the server is up, then you can continue using git bash for everything else. After stripe login, you need to enter stripe apps start. Once you do that, the server is up and running and you can go back to using git bash or any other tool. 2\. Install Dependencies We'll be using an extra package for QR code generation. Install it using npm: ` 3\. Set Up the Main Component Let's look at the home.tsx file, where we'll use Stripe's UI components: ` These components are similar to other UI libraries like Bootstrap or Tailwind CSS. 4\. Create the UI Structure Our app will have: * An input field for the URL * Validation using a regex pattern * Error handling for invalid URLs * QR code generation and display Here is the Home.tsx file that is located in the src/views folder ` * ContextView` is at the top level of the app where we see the Title and the link to the Stripe Docs that we placed in our Context View. * Box is how you use Divs. * Banners can be used to show notification errors or any other item you wish to display. * Textfields are input fields. * Everything else is pretty self-explanatory. 5\. Handle Content Security Policy One problem I personally ran into was when I tried to redirect users, the Stripe policies would block it since I did not express that I knew what it was doing. I had to go into the stripe-app.json file and mention the specific security policies. For this particular exercise, I kept these as null. This is my stripe-app.json file. ` 6\. Configure App Views As you can see here, the stripe-app.json file shows the views for each file I have. The Home.tsx file and the Invoice.tsx are also included This is our way of saying that for each view we have, show the app functionality on that page. Our stripe-app.json file will show it but also, the manifest.js file in our .build folder will also show the same. Any view that doesn't have a file will not show the application's functionality. So, if I were to go to transactions, the app would not show the same logic as the home or invoices page. By following these steps, you'll have a fully functional QR code generator app for Stripe. This is just a simple example, but the potential for Stripe apps is massive, especially for businesses serving e-commerce customers. If you need help or get stuck, don't hesitate to reach out, danny.thompson@thisdot.co. The Stripe team is also very active in answering questions, so leverage them as a resource. Happy coding!...

Dec 18, 2024

4 mins

stripe

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Getting started with 3D and React

Introduction

Install

Create your first sense

Cameras

Events

Loading models

Animation

Mark Shenouda

You might also like

Making Seamless Page Transitions with the View Transitions API

How to build an AI assistant with OpenAI, Vercel AI SDK, and Ollama with Next.js

Build Beautiful Storefronts Quickly with Shopify and Next

Building a Stripe App: A Step-by-Step Guide to QR Code Generation

Let's innovate together!

You might also like

Making Seamless Page Transitions with the View Transitions API

How to build an AI assistant with OpenAI, Vercel AI SDK, and Ollama with Next.js

Build Beautiful Storefronts Quickly with Shopify and Next

Building a Stripe App: A Step-by-Step Guide to QR Code Generation