General

Dismantling Your AI Bias with Jerome Hardaway and Tracy Lee

Published Mar 12, 2024

Updated Mar 12, 2024

1 min read

In this inaugural episode in a series on the six steps to AI adoption, Tracy Lee and Jerome Hardaway explore the impact of AI on various industries, emphasizing the need to address bias and adapt as developers. The first step is dismantling your own bias against AI.

They advocate for treating AI as a tool to enhance human capabilities, and how it can revolutionize education and streamline workflows by augmenting our everyday tasks. Even better, the transformative potential of AI creates new job opportunities, necessitating education and upskilling initiatives to prepare individuals for the changing job market.

By addressing bias, embracing continuous learning, and recognizing AI's capacity to augment human capabilities, we can unlock its full potential in shaping a better future.

Download this podcast episode here.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

Detect Hand Sign Languages with Tensorflow

Interested in learning how to use Tensorflow to detect hand sign languages in your apps? By the end of this read, you will know how to implement Tensorflow in your application with very simple steps. In our example today, we will be using Vue. What is Tensorflow? Tensorflow is an end-to-end platform _(meaning: delivering complex systems or services in functional form after developing it from beginning to end.)_ used for building Machine Learning applications, and it is also open-source. TensorFlow enables you to build dataflow graphs and structures to define how data moves through a graph by taking inputs as a multi-dimensional array called Tensor. You can read more on Tensorflow here. What is a Model? A model is a function with learnable parameters that maps an input to an output. A well-trained model will provide an accurate mapping from the input to the desired output. Tensorflow Models Tensorflow models are pre-trained models, and there are four defined categories of them: - Vision: Analyze features in images and videos. - Body: Detect key points and poses on the face, hands, and body with models from MediPipe - Text: Enable NLP in your web app using the power of BERT and other Transformer encoder architectures. - Audio: Classify audio to detect sounds. If you want to go into more detail, check out Tensorflow Models. All these models are broken down into subs and for our case, we will be making use of the Body Model which has the hand pose detection we need in order to detect the hand signs. Hand Pose Detection This model used a 2D and 3D multi-dimensional array which enables it to predict the keypoints of the hands. Example of a 2D is [[1,2],[3,5],[7,8],[20,44]] and that of a 3D is [[1,2,5],[3,5,8],[7,8,6],[20,44,100]]. This hand pose detection is a model from the MediPipe as we established above, and it provides us with two model types which are lite and full. The accuracy of the prediction increases from lite to full while the inference speed reduces, i.e. the response time will be slower as the accuracy increases. What do we need? There are a few dependencies we need to get things working, and I also will be assuming that you have your project set up as well. You will need to add these dependencies to the project ` Above, in the commands, you will notice we added a fingerpose. Let's talk a little about what we need the figerpose for. Fingerpose Fingerpose is a gesture classifier for hand landmarks detected by Mediapipe hand pose detection. It also allows you to add your own hand gesture, which means that a gesture that signifies the letter Z can signify Hello based on your fingerpose data. We will see an example of how the data looks in a bit. You can check out fingerpose for more details. Get started We are going to use Vue for this illustration. We will start by looking at the HTML first, and then we will cover the JavaScript. Our Template will be a basic HTML that will have a video tag so we can show a video after getting access to our webcam. Template ` The snippet above shows a div and a video tab. The video is used when we gain access to the webcam. We will now be writing the JS required to initialize the webcam. Script ` We imported two methods from vue: onMounted and ref. The onMounted runs when the page is fully mounted while the ref is used to declare a reactive value to reference the video element. If you look at the video tag in the template, you will notice a ref property. You can check out Template ref and onMounted lifecycle hook. In the openCam function, we first try to test if mediaDevices is available on your browser navigation. >The MediaDevices interface provides access to connected media input devices like cameras and microphones, as well as screen sharing. In essence, it lets you obtain access to any hardware source of media data. This MediaDevice has a method getUserMedia which prompts the user for permission to use a media input. You can find all you need to know about getUserMedia here. From the snippet, we can see that getUserMedia returns a promise, and with that, we can get the media stream as a response using then(). We check if the video element has srcObject or not. If it does we assign the media stream to the srcObject and if not, we convert the media stream to a URL and assign it to the src of the video element. With this Snippet and with a few style, you should have your video showing your awesome face! Introducing Tensorflow and Hand Detection Now that we got our webcam working, we will update the Template and the script in order to detect, predict, and display the alphabet based on the hand sign prediction. The updated HTML should now look like this: ` The div with class name alphabet will display the alphabet based on the hand sign prediction. We will be introducing two(2) new functions, createDetectionInstance and handleSignDetection. Firstly, lets begin with the createDetectionInstance which is an integral part of the hand sign detection and then we will introduce handleSignDetection which predicts and displays the hand sign. ` To be able to detect hand poses, we need to create an instance of the handpose detector, and here, we created a function createDetectionInstance which is an asynchronous function. You can check out this Tensorflow blog to see more details. Now that we have created an avenue to detect hand signs, let us start detecting the hand. In that light, we will be adding a handleSignDetection function. ` The handleSignDetection runs after creating the detection instance. We have a setInterval that runs every 2 seconds (PS: _the 2 seconds timing is arbitrary and can be less or more_) to check if there is any hand sign. We also have a conditional statement to ensure the video element exists, and the detection instance was created accordingly. So, the detector calls a method estimateHands, which tries to predict the hand pose by getting keypoints with values that are either in 2D or 3D (Multi-dimensional Array). If you check your console log, you will see an array of data if any hand pose is detected. Now that we can detect hand poses, we will now add fingerpose that will help predict and display the alphabet based on the hand sign. ` Assuming that our detector sensed a hand, it is time to match this value based on the hand signs we created with the fingerpose. The landmark variable is a 3D array pulled from the hand result's keypoint3D key value. There is also a keypoint as well, which is a 2D value, and both will give the same result. Now, using GE.estimate, we can generate a possible gesture that matches the sign, and a score/confidence is assigned to each gesture pending the amount of gesture predicted. So, the gesture with the highest score/confidence is selected since it is estimated to be the closest to the hand sign from the figerpose hand signs we created. We also imported Handsigns and its content looks like this: You can also get the handsigns folder from the 100-ms-vue repository. Looking at the screenshot, there is a GestureDescription instance that takes a string A which will represent what the hand sign will stand for. So, it could be anything you want the handsign to stand for. >onMounted is asynchronous because we need to ensure that our detection instance is created, which is required to detect the hand sign. With the updated code, you should be able to display some letters. Conclusion Don't forget, you can see in detail how this was implemented in one of This Dot Labs' open source projects 100-ms-vue. Please note that what we did is just a basic implementation, and to have a production-ready version, it will need a bigger model, and a more complex detection to be able to identify hand sign language....

Jul 12, 2023

7 mins

Artificial IntelligenceTensorflowMachine Learning

Svelte 5 is Here!

Svelte 5 was finally released after a long time in development. Fortunately, we've been able to test it for some time, and now it has a stable release. Let's dig into its features and why this is such a significant change, even though Svelte 4 code is almost 100% compatible. Svelte syntax everywhere This is one of my favorite additions. Previously, Svelte syntax was limited to a component element. Refactoring code from a component and moving it to a JavaScript file worked differently. Now you can use the .svelte.js or .svelte.ts extension, which allows you to use the same syntax everywhere. It's important to note that it's a way to express that this is not just JS, and what you write may be compiled to something else, just like .svelte files are not just html even though they look very similar. Runes The introduction of runes is one of the most significant changes in Svelte. Many users felt attracted to variables being instantly reactive in previous versions. ` There was, however, a lot of magic underneath and a considerable amount of work from the compiler to make it behave reactively. $state In Svelte 5, a reactive variable has to be explicitly declared using the $state rune, which brings a clear distinction between what is reactive and what isn’t. ` In the previous code, $state is not an actual function being called; it's a hint for the compiler to do something special with this declaration. Rune names always start with a dollar sign ($) and do not need to be imported. However, there’s much more to the changes than just the way we declare reactive variables. Runes bring a lot of new features that make Svelte 5 a great improvement. In Svelte 5, objects or arrays use Proxies to allow for granular reactivity, meaning that individual properties in an object are reactive, even if they are nested objects or arrays. If you modify a property, it will only update that property and will not trigger an update on the whole object. It also supports triggering updates when methods like array.push are called. In previous versions, an assignment was required to trigger an update: ` To have a similar behavior of svelte 4 (no deep reactivity), use the $state.raw() rune. This syntax of .* is common for related features of a rune: $state.raw() will require an assignment to trigger reactivity. ` Because proxies are used, you may need to extract the underlying state instead of the proxy itself. In that case, $state.snapshot() should be used: ` $derived We will use $derived and $derived.by to declare a derived state. The two runes are essentially the same, except one allows us to use a function instead of an expression, allowing for more complex operations to calculate the derived values. ` $effect Effects are useful for running something triggered by a change in state. One of the things you'll note from the $effect rune is that dependencies don't need to be explicit. Those will be picked from reactive values read synchronously in the body of the effect function. ` Something to bear in mind is that these dependencies are picked each time the effect runs. The values read during the last run become the dependencies of the effect. ` Depending on the result of the random method, foo or bar will stop being a dependency of the effect. You should place them outside the condition so they can trigger reruns of the effect. *Variants* of the effect rune are $effect.pre, which runs before a DOM update, and $effect.tracking(), which checks for the context of the effect (true for an effect inside an effect or in the template). $props This rune replaces the export let keywords used in previous versions to define a component's props. To use the $props syntax, we can consider it to return an object with all the properties a component receives. We can use JavaScript syntax to destructure it or use the rest property if we want to retrieve every property not explicitly destructured.. ` $bindable If you want to mutate a prop so that the change flows back to the parent, you can use the $bindable prop to make it work in both directions. A parent component can pass a value to a child component, and the child component is able to modify it, returning the data back to the parent component. ` $inspect The $inspect rune only works in dev mode and will track dependencies deeply. By default, it will call console.log with the provided argument whenever it detects a change. To change the underlying function, use $inspect.with().with): ` $host The host rune gives access to the host element when compiling as a custom element: ` ` Other changes Another important change is that component events are just props in Svelte 5, so you can destructure them using the $props() rune: ` A special children property can be used to project content into a component instead of using slots. Inside the components, use the [@render[(https://svelte.dev/docs/svelte/@render) tag to place them. ` Optional chaining (?.) prevents from attempting to render it if no children are passed in. If you need to render different components in different places (named slots before Svelte 5, you can pass them as any other prop, and use @render with them. Snippets Snippets allow us to declare markup slices and render them conveniently using the render tag (@render). They can take any number of arguments. ` Snippets can also be passed as props to other components. ` ` Conclusion Some exciting changes to the Svelte syntax were introduced while, at the same time, maximum compatibility efforts were made. This was a massive rewrite with numerous improvements in terms of performance, bundle size, and DX. Besides that, a new CLI has been released, making the whole experience of starting a project or adding features delightful. If you haven't tried Svelte before, it's a great time to try it now....

Nov 27, 2024

4 mins

Svelte

The HTML Dialog Element: Enhancing Accessibility and Ease of Use

The HTML Dialog Element: Enhancing Accessibility and Ease of Use Dialogs are a common component added to applications, whether on the web or in native applications. Traditionally there has not been a standard way of implementing these on the web, resulting in many ad-hoc implementations that don’t act consistently across different web applications. Often, commonly expected features are missing from dialogs due to the complexity of implementing them. However, web browsers now offer a standard dialog element. Why use the dialog element? The native dialog element streamlines the implementation of dialogs, modals, and other kinds of non-modal dialogs. It does this by implementing many of the features needed by dialogs for you that are already baked into the browser. This is helpful as it reduces the burden on the developer when making their applications accessible by ensuring that user expectations concerning interaction are met, and it can also potentially simplify the implementation of dialogs in general. Basic usage Adding a dialog using the new tag can be achieved with just a few lines of code. ` However, adding the dialog alone won’t do anything to the page. It will show up only once you call the .showModal() method against it. ` Then if you want to close it you can call the .close() method on the dialog, or press the escape key to close it, just like most other modals work. Also, note how a backdrop appears that darkens the rest of the page and prevents you from interacting with it. Neat! Accessibility and focus management Correctly handling focus is important when making your web applications accessible to all users. Typically you have to move the current focus to the active dialog when showing them, but with the dialog element that’s done for you. By default, the focus will be set on the first focusable element in the dialog. You can optionally change which element receives focus first by setting the autofocus attribute on the element you want the focus to start on, as seen in the previous example where that attribute was added to the close element. Using the .showModal() method to open the dialog also implicitly adds the dialog ARIA role to the dialog element. This helps screen readers understand that a modal has appeared and the screen so it can act accordingly. Adding forms to dialogs Forms can also be added to dialogs, and there’s even a special method value for them. If you add a element with the method set to dialog then the form will have some different behaviors that differ from the standard get and post form methods. First off, no external HTTP request will be made with this new method. What will happen instead is that when the form gets submitted, the returnValue property on the form element will be set to the value of the submit button in the form. So given this example form: ` The form element with the example-form id will have its returnValue set to Submit. In addition to that, the dialog will close immediately after the submit event is done being handled, though not before automatic form validation is done. If this fails then the invalid event will be emitted. You may have already noticed one caveat to all of this. You might not want the form to close automatically when the submit handler is done running. If you perform an asynchronous request with an API or server you may want to wait for a response and show any errors that occur before dismissing the dialog. In this case, you can call event.preventDefault() in the submit event listener like so: ` Once your desired response comes back from the server, you can close it manually by using the .close() method on the dialog. Enhancing the backdrop The backdrop behind the dialog is a mostly translucent gray background by default. However, that backdrop is fully customizable using the ::backdrop pseudo-element. With it, you can set a background-color to any value you want, including gradients, images, etc. You may also want to make clicking the backdrop dismiss the modal, as this is a commonly implemented feature of them. By default, the <dialog> element doesn’t do this for us. There are a couple of changes that we can make to the dialog to get this working. First, an event listener is needed so that we know when the user clicks away from the dialog. ` Alone this event listener looks strange. It appears to dismiss the dialog whenever the dialog is clicked, not the backdrop. That’s the opposite of what we want to do. Unfortunately, you cannot listen for a click event on the backdrop as it is considered to be part of the dialog itself. Adding this event listener by itself will effectively make clicking anywhere on the page dismiss the dialog. To correct for this we need to wrap the contents of the dialog content with another element that will effectively mask the dialog and receive the click instead. A simple element can do! ` Even this isn’t perfect though as the contents of the div may have elements with margins in them that will push the div down, resulting in clicks close to the edges of the dialog to dismiss it. This can be resolved by adding a couple of styles the the wrapping div that will make the margin stay contained within the wrapper element. The dialog element itself also has some default padding that will exacerbate this issue. ` The wrapping div can be made into an inline-block element to contain the margin, and by moving the padding from the parent dialog to the wrapper, clicks made in the padded portions of the dialog will now interact with the wrapper element instead ensuring it won’t be dismissed. Conclusion Using the dialog element offers significant advantages for creating dialogs and modals by simplifying implementation with reasonable default behavior, enhancing accessibility for users that need assistive technologies such as screen readers by using automatic ARIA role assignment, tailored support for form elements, and flexible styling options....

Oct 25, 2024

4 mins

HTMLJavaScript

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Dismantling Your AI Bias with Jerome Hardaway and Tracy Lee

You might also like

Detect Hand Sign Languages with Tensorflow

Svelte 5 is Here!

The HTML Dialog Element: Enhancing Accessibility and Ease of Use

Let's innovate together!

You might also like

Detect Hand Sign Languages with Tensorflow

Svelte 5 is Here!

The HTML Dialog Element: Enhancing Accessibility and Ease of Use