General

Git Strategies for Working on Teams

Published Sep 1, 2022

Updated Feb 13, 2023

4 min read

This article was written over 18 months ago and may contain information that is out of date. Some content may be relevant but please refer to the relevant official documentation or available resources for the latest information.

Background

Every team I’ve worked on has had different expectations and rules around git. Some had strict guidelines that created developer experience issues. Others had rules that were so loose that there was no consistency on the team.

These are my thoughts on what I’ve discovered to be a healthy balance between the strict and loose rules on teams. I hope to make suggestions for your team, and process that don’t sound dogmatic, and eventually help.

Always Branch and Pull Request to Main

This may seem like a no brainer to some, but no one should ever be interacting on your repo’s main branch directly. All changes to upstream main should be handled via pull requests (PR) with the exception of the initial repository commit. This will make all the suggestions throughout the article function properly. But it should also make it so your main branch will remain healthy assuming you have CI/CD working on PR. It also enables peer review on work which is a good habit. I still PR on my personal projects to make so I understand a set of changes in context as well.

Merges to main via Squash and Merge

All git hosting services have a “Squash and Merge” button on PRs which is an alias for:

git checkout main
git merge --squash <branch>
git commit

This takes all the commits on the specified branch and reduces them into a single atomic commit that gets inserted into the main branch. This is great for a few reasons:

Keeps your git history clean
1. omits merge commits
2. keeps merge historically fully sequential
Creates a single commit describing a bulk of changes but still makes the history of those changes available via a link to the original PR for full context
Helps using bisect strategies to identify when a change was introduced, and reference the PR in which that change was introduced

The following is an example output of the squash and merge strategy:

Individual Branching

I have two rules I suggest for individual branches:

conform to some naming convention the team sets
use a non-destructive strategy for commits and updates once a review has occurred

Rule 1 exists for a variety of reasons but primarily to avoid naming collisions. Rule 2, on the other hand, exists for a reviewer’s sake. All net new changes should be done via commits once a review has be initiated. Otherwise, it’s hard for a reviewer to track what they’ve already looked at versus what’s changed. Be nice to your reviewers. Once that PR is open, you may not use --amend or rebase on existing commits moving forward.

Otherwise, how you keep your branch up-to-date with the main branch is entirely your choice. Merge is typically the easiest if there are conflicts, especially when trying to resolve, and is usually best for novice developers. Rebase is great if you don’t have any merge conflicts or merge commits, and lets you get all the latest changes in the main brach. At the end of the day, do what is best for you.

The squash and merge strategy makes it so your branch can be whatever you want it to be. Do you commit every 5 minutes? Fine! Have 15 merge commits from main? That’s also fine! All these actions tell a story but squash and merge to main keeps that history on your branch, and PR and does not impact upstream main.

Long Running Epic Branches

All my rules above are great for single changes applied directly to main. However, some teams work on long running epic branches to avoid introducing changes to main that are not ready for release. The best way to avoid long running epic branches is feature flags, but some teams aren’t able to use these for one reason or another.

That’s ok! For this situation, I recommend the following:

Create a base integration branch that is kept up with main using the merge strategy
Create individual branches off the integration branch and follow my instructions on individual branching from above
Squash merge branches back into the integration
Squash merge the integration back into main when ready

I’ve been on teams that try to do fancy rebase --onto strategies with the integration branch, and then the following individual branches. At the end of the day, this creates a lot of unnecessary git work for teams, and wastes time that is better spent working on features, bugs, and tech debt. This is because branches have to be updated in a particular way, and sometimes the conflicts that arise lead to lost changes and duplicated effort and work. Simplify your process and simplify your teams’ lives.

Conclusion

This only covers a few of the most common situations, but these general rules should help for teams and individuals in the way that is best for them while keeping the upstream main branch healthy and easier to debug when critical issues are identified.

I hope this helps you and your team on your next project and allows more time for technical debt and other important work that may have been lost to git related issues.

This Dot is a consultancy dedicated to guiding companies through their modernization and digital transformation journeys. Specializing in replatforming, modernizing, and launching new initiatives, we stand out by taking true ownership of your engineering projects.

We love helping teams with projects that have missed their deadlines or helping keep your strategic digital initiatives on course. Check out our case studies and our clients that trust us with their engineering.

About the author(s)

Dustin Goodman
Engineering Manager with a passion for web and application development. Speaker and writer on work experiences and software development. Dog dad and musician fueled by coffee.
@dustinsgoodman @dustinsgoodman

"How do I undo my most recent commit?" - Mastering the git reset command

"How do I undo my most recent commit?" - Mastering the git reset command Git is a powerful version control system, but even experienced developers sometimes make mistakes. Whether you've committed changes to the wrong branch, made a typo in your commit message, or simply want to undo recent changes, the git reset command is your go-to solution. In this post, we'll explore how to use git reset to manage your commit history effectively. The Basics of git reset At its core, git reset moves the HEAD and current branch pointer to a specified commit, effectively "resetting" your working directory to a previous state. The basic syntax is: ` Where specifies how to handle changes in the working directory and staging area, and is the commit you want to reset to. Soft vs. Hard Reset: Understanding the Difference Git reset has two main modes: --soft and --hard. Let's explore each: Soft Reset (--soft) A soft reset moves your HEAD to the specified commit but keeps your changes staged. This is useful when you want to undo commits but keep the changes for recommitting. Example: ` This command undoes the last commit, keeping the changes staged. You can now make additional changes or recommit with a new message. Hard Reset (--hard) A hard reset is more drastic. It moves your HEAD to the specified commit and discards all changes after that commit. Example: ` This command undoes the last commit and discards all changes. Be cautious with this command, as it can lead to data loss! Specifying a Commit Hash Instead of using relative references like HEAD~1, you can specify an exact commit hash: ` This resets to the commit with hash 1a2b3c4, keeping changes staged. Using HEAD~n Notation The HEAD~n notation allows you to specify how many commits back you want to reset. For example: ` This resets to three commits before the current HEAD, discarding all changes. Reverting Pushed Changes If you've already pushed commits you want to undo, you should use git revert instead of git reset to avoid rewriting public history. However, if you must use git reset on a shared branch, you'll need to force push: ` Be extremely cautious with force pushing, as it can cause issues for other developers. Generic Usage of git reset git reset can also be used to unstage changes: ` This removes file.txt from the staging area without changing its contents. Conclusion The git reset command is a powerful tool for managing your Git history. Whether you're undoing recent commits, unstaging changes, or completely resetting your working directory, understanding the different modes of git reset can help you navigate tricky situations in your Git workflow. Remember to use git reset with caution, especially when working with shared repositories. When in doubt, create a backup branch before performing any reset operations....

Oct 18, 2024

2 mins

Git

Mastering Git Rerere: Solving Repetitive Merge Conflicts with Ease

Mastering Git Rerere: Solving Repetitive Merge Conflicts with Ease Introduction: Git, the popular version control system, has revolutionized how developers collaborate on projects. However, one common pain point in Git is repetitive merge conflicts. Fortunately, Git provides a powerful and not-so-well-known solution called git rerere (reuse recorded resolution) that can save you time and effort when resolving conflicts. In this blog post, we will explore how to configure and use git rerere, and demonstrate its effectiveness in solving merge conflicts. Understanding Git Rerere: Git rerere is a feature that allows Git to remember how you resolved a particular merge conflict in a particular file and automatically apply the same resolution in the future. It works by recording the conflict resolution in a hidden directory called .git/rr-cache. This way, when Git encounters the same conflict in the same file in the future, it can reuse the recorded resolution, saving you from manually resolving the conflict again. Configuring Git Rerere: Before using git rerere, you need to enable it in your Git configuration; only once git rerere has been enabled, Git will start recording and remembering resolved conflicts.Open your terminal and run the following command: ` This command enables git rerere globally, making it available for all your repositories. You can also enable it per-repository by omitting the --global flag. How to use it: Let's start with a really easy example, then describe a couple of use cases. We have merged a branch (branch-one) into our main, and we are working on two different features in two different branches (branch-two and branch-three). We need to rebase our branches with main, so we start with ` It turns out that there are some conflicts on App.tsx file: photo1.png We solve all the conflicts and finish with the rebase and push. As you can see, there are an extra line in the rebase output message that says: This means that thanks to rerere option enabled, we have saved in our project's .git/rr-cache this resolution for this particular conflict. Now let's switch branches into branch-three cause we want to rebase on main also this one: ` It seems that we have the same conflicts here too, but this time, on the rebase output message, we can read: The conflict has been resolved automatically; if we check our IDE, we can see the change (and check if it works for us) ready to be committed and pushed with the same resolution we manually used in the past rebase. The example above focuses on a rebase, but of course, git rerere also works for conflicts that came out from a merge command. Here are a couple of real-life scenarios where git rerere can save the day: Frequent Integration of Feature Branches: Imagine you're working on a feature branch that frequently needs to be merged into the main development branch. Each time you merge, you encounter the same merge conflicts. With git rerere, you only need to resolve these conflicts once. After that, Git remembers the resolutions and automatically applies them in future merges, saving you from resolving the same conflicts repeatedly. Reapplying Patches or Fixes: Let's say you have a situation where you need to apply the same set of changes or fixes to multiple branches. When you encounter conflicts during this process, git rerere can remember how you resolved them the first time. Then, when you apply the changes to other branches, Git can automatically reuse the recorded resolutions, sparing you from manually resolving the same conflicts repeatedly. Benefits of Git Rerere: Git rerere offers several benefits that make it a valuable tool for developers, regardless of whether you prefer git merge or git rebase: 1. Time-saving: By reusing recorded resolutions, git rerere eliminates the need to manually resolve repetitive merge conflicts, saving you valuable time and effort. 2. Consistency: Git rerere ensures consistent conflict resolutions across multiple merges or rebases, reducing the chances of introducing errors or inconsistencies. 3. Improved productivity: With git rerere, you can focus on more critical tasks instead of getting stuck in repetitive conflict resolutions. Conclusion: Git rerere is a powerful feature that simplifies resolving repetitive merge conflicts. By enabling git rerere and recording conflict resolutions, you can save time, improve productivity, and ensure consistent conflict resolutions across your projects. Incorporate git rerere into your Git workflow, and say goodbye to the frustration of repetitive merge conflicts....

Jan 5, 2024

4 mins

Git

The Dangers of ORMs and How to Avoid Them

Background We recently engaged with a client reporting performance issues with their product. It was a legacy ASP .NET application using Entity Framework to connect to a Microsoft SQL Server database. As the title of this post and this information suggest, the misuse of this Object-Relational Mapping (ORM) tool led to these performance issues. Since our blog primarily focuses on JavaScript/TypeScript content, I will leverage TypeORM to share examples that communicate the issues we explored and their fixes, as they are common in most ORM tools. Our Example Data If you're unfamiliar with TypeORM I recommend reading their docs or our other posts on the subject. To give us a variety of data situations, we will leverage a classic e-commerce example using products, product variants, customers, orders, and order line items. Our data will be related as follows: The relationship between customers and products is one we care about, but it is optional as it could be derived from customer->order->order line item->product. The TypeORM entity code would look as follows: ` With this example set, let's explore some common misuse patterns and how to fix them. N + 1 Queries One of ORMs' superpowers is quick access to associated records. For example, we want to get a product and its variants. We have 2 options for writing this operation. The first is to join the table at query time, such as: ` Which resolves to a SQL statement like (mileage varies with the ORM you use): ` The other option is to query the product variants separately, such as: ` This example executes 2 queries. The join operation performs 1 round trip to the database at 200ms, whereas the two operations option performs 2 round trips at 100ms. Depending on the location of your database to your server, the round trip time will impact your decision here on which to use. In this example, the decision to join or not is relatively unimportant and you can implement caching to make this even faster. But let's imagine a different situation/query that we want to run. Let's say we want to fetch a set of orders for a customer and all their order items and the products associated with them. With our ORM, we may want to write the following: ` When written out like this and with our new knowledge of query times, we can see that we'll have the following performance: 100ms * 1 query for orders + 100ms * order items count + 100ms * order items' products count. In the best case, this only takes 100ms, but in the worst case, we're talking about seconds to process all the data. That's an O(1 + N + M) operation! We can eagerly fetch with joins or collection queries based on our entity keys, and our query performance becomes closer to 100ms for orders + 100ms for order line items join + 100ms for product join. In the worst case, we're looking at 300ms or O(1)! Normally, N+1 queries like this aren't so obvious as they're split into multiple helper functions. In other languages' ORMs, some accessors look like property lookups, e.g. order.orderItems. This can be achieved with TypeORM using their lazy loading feature (more below), but we don't recommend this behavior by default. Also, you need to be wary of whether your ORM can be utilized through a template/view that may be looping over entities and fetching related records. In general, if you see a record being looped over and are experiencing slowness, you should verify if you've prefetched the data to avoid N+1 operations, as they can be a major bottleneck for performance. Our above example can be optimized by doing the following: ` Here, we prefetch all the needed order items and products and then index them in a hash table/dictionary to look them up during our loops. This keeps our code with the same readability but improves the performance, as in-memory lookups are constant time and nearly instantaneous. For performance comparison, we'd need to compare it to doing the full operation in the database, but this removes the egregious N+1 operations. Eager v Lazy Loading Another ORM feature to be aware of is eager loading and lazy loading. This implementation varies greatly in different languages and ORMs, so please reference your tool's documentation to confirm its behavior. TypeORM's eager v lazy loading works as follows: - If eager loading is enabled for a field when you fetch a record, the relationship will automatically preload in memory. - If lazy loading is enabled, the relationship is not available by default, and you need to request it via a key accessor that executes a promise. - If neither is enabled, it defaults to the behavior ascribed above when we explained handling N+1 queries. Sometimes, you don't want these relationships to be preloaded as you don't use the data. This behavior should not be used unless you have a set of relations that are always loaded together. In our example, products likely will always need product variants loaded, so this is a safe eager load, but eager loading order items on orders wouldn't always be used and can be expensive. You also need to be aware of the nuances of your ORM. With our original problem, we had an operation that looked like product.productVariants.insert({ … }). If you read more on Entity Framework's handling of eager v lazy loading, you'll learn that in this example, the product variants for the product are loaded into memory first and then the insert into the database happens. Loading the product variants into memory is unnecessary. A product with 100s (if not 1000s) of variants can get especially expensive. This was the biggest offender in our client's code base, so flipping the query to include the ID in the insert operation and bypassing the relationship saved us _seconds_ in performance. Database Field Performance Issues Another issue in the project was loading records with certain data types in fields. Specifically, the text type. The text type can be used to store arbitrarily long strings like blog post content or JSON blobs in the absence of a JSON type. Most databases use a technique that stores text fields off rows, which requires a special file system lookup operation to fetch that data. This can make a typical database lookup that would take 100ms under normal conditions to take 200ms. If you combine this problem with some of the N+1 and eager loading problems we've mentioned, this can lead to seconds, if not minutes, of query slowdown. For these, you should consider not including the column by default as part of your ORM. TypeORM allows for this via their hidden columns feature. In this example, for our product description, we could change the definition to be: ` This would allow us to query products without descriptions quickly. If we needed to include the product description in an operation, we'd have to use the addSelect function to our query to include that data in our result like: ` This is an optimization you should be wary of making in existing systems but one worth considering to improve performance, especially for data reports. Alternatively, you could optimize a query using the select method to limit the fields returned to those you need. Database v. In-Memory Fetching Going back to one of our earlier examples, we wrote the following: ` This involves loading our data in memory and then using system memory to perform a group-by operation. Our database could have also returned this result grouped. We opted to perform this operation like this because it made fetching the order item IDs easier. This takes some performance challenges away from our database and puts the performance effort on our servers. Depending on your database and other system constraints, this is a trade-off, so do some benchmarking to confirm your best options here. This example is not too bad, but let's say we wanted to get all the orders for a customer with an item that cost more than $20. Your first inclination might be to use your ORM to fetch all the orders and their items and then filter that operation in memory using JavaScript's filter method. In this case, you're loading data you don't need into memory. This is a time to leverage the database more. We could write this query as follows in SQL: ` This just loads the orders that had the data that matched our condition. We could write this as: ` We constrained this to a single customer, but if it were for a set of customers, it could be significantly faster than loading all the data into memory. If our server has memory limitations, this is a good concern to be aware of when optimizing for performance. We noticed a few instances on our client's implementation where the filter operations were applied in functions that appeared to run the operation in the database but were running the operation in memory, so this was preloading more data into memory than needed on a memory-constrained server. Refer to your ORM manual to avoid this type of performance hit. Lack of Indexes The final issue we encountered was a lack of indexes on key lookups. Some ORMs do not support defining indexes in code and are manually applied to databases. These tend not to be documented, so an out-of-sync issue can happen in different environments. To avoid this challenge, we prefer ORMs that support indexes in code like TypeORM. In our last example, we filtered on the cost of an order item, but the cost field does not contain an index. This leads to a full table scan of our data collection filtered by the customer. The query cost can be very expensive if a customer has thousands of orders. Adding the index can make our query super fast, but it comes at a cost. Each new index makes writing to the database slower and can exponentially increase the size of our database needs. You should only add indexes to fields that you are querying against regularly. Again, be sure you can notate these indexes in code so they can be replicated across environments easily. In our client's system, the previous developer did not include indexes in the code, so we retroactively added the database indexes to the codebase. We recommend using your database's recommended tool for inspection to determine what indexes are in place and keep these systems in sync at all times. Conclusion ORMs can be an amazing tool to help you and your teams build applications quickly. However, they have gotchas for performance that you should be aware of and can identify while developing or during code reviews. These are some of the most common examples I could think of for best practices. When you hear the horror stories about ORMs, these are some of the challenges typically discussed. I'm sure there are more, though. What are some that you know? Let us know!...

Jul 3, 2024

8 mins

Architectural DesignDesign PatternsArchitecture

D1 SQLite: Writing queries with the D1 Client API

Writing queries with the D1 Client API In the previous post we defined our database schema, got up and running with migrations, and loaded some seed data into our database. In this post we will be working with our new database and seed data. If you want to participate, make sure to follow the steps in the first post. We’ve been taking a minimal approach so far by using only wrangler and sql scripts for our workflow. The D1 Client API has a small surface area. Thanks to the power of SQL, we will have everything we need to construct all types of queries. Before we start writing our queries, let's touch on some important concepts. Prepared statements and parameter binding This is the first section of the docs and it highlights two different ways to write our SQL statements using the client API: prepared and static statements. Best practice is to use prepared statements because they are more performant and prevent SQL injection attacks. So we will write our queries using prepared statements. We need to use parameter binding to build our queries with prepared statements. This is pretty straightforward and there are two variations. By default we add ? ’s to our statement to represent a value to be filled in. The bind method will bind the parameters to each question mark by their index. The first ? is tied to the first parameter in bind, 2nd, etc. I would stick with this most of the time to avoid any confusion. ` I like this second method less as it feels like something I can imagine messing up very innocently. You can add a number directly after a question mark to indicate which number parameter it should be bound to. In this exampl, we reverse the previous binding. ` Reusing prepared statements If we take the first example above and not bind any values we have a statement that can be reused: ` Querying For the purposes of this post we will just build example queries by writing them out directly in our Worker fetch handler. If you are building an app I would recommend building functions or some other abstraction around your queries. select queries Let's write our first query against our data set to get our feet wet. Here’s the initial worker code and a query for all authors: ` We pass our SQL statement into prepare and use the all method to get all the rows. Notice that we are able to pass our types to a generic parameter in all. This allows us to get a fully typed response from our query. We can run our worker with npm run dev and access it at http://localhost:8787 by default. We’ll keep this simple workflow of writing queries and passing them as a json response for inspection in the browser. Opening the page we get our author results. joins Not using an ORM means we have full control over our own destiny. Like anything else though, this has tradeoffs. Let’s look at a query to fetch the list of posts that includes author and tags information. ` Let’s walk through each part of the query and highlight some pros and cons. ` * The query selects all columns from the posts table. * It also selects the name column from the authors table and renames it to author_name. * It aggregates the name column from the tags table into a JSON array. If there are no tags, it returns an empty JSON array. This aggregated result is renamed to tags. ` * The query starts by selecting data from the posts table. * It then joins the authors table to include author information for each post, matching posts to authors using the author_id column in posts and the id column in authors. * Next, it left joins the posts_tags table to include tag associations for each post, ensuring that all posts are included even if they have no tags. * Next, it left joins the tags table to include tag names, matching tags to posts using the tag_id column in posts_tags and the id column in tags. * Finally, group the results by the post id so that all rows with the same post id are combined in a single row SQL provides a lot of power to query our data in interesting ways. JOIN ’s will typically be more performant than performing additional queries.You could just as easily write a simpler version of this query that uses subqueries to fetch post tags and join all the data by hand with JavaScript. This is the nice thing about writing SQL, you’re free to fetch and handle your data how you please. Our results should look similar to this: ` This brings us to our next topic. Marshaling / coercing result data A couple of things we notice about the format of the result data our query provides: Rows are flat. We join the author directly onto the post and prefix its column names with author. ` Using an ORM we might get the data back as a child object: ` Another thing is that our tags data is a JSON string and not a JavaScript array. This means that we will need to parse it ourselves. ` This isn’t the end of the world but it is some more work on our end to coerce the result data into the format that we actually want. This problem is handled in most ORM’s and is their main selling point in my opinion. insert / update / delete Next, let’s write a function that will add a new post to our database. ` There’s a few queries involved in our create post function: * first we create the new post * next we run through the tags and either create or return an existing tag * finally, we add entries to our post_tags join table to associate our new post with the tags assigned We can test our new function by providing post content in query params on our index page and formatting them for our function. ` I gave it a run like this: http://localhost:8787authorId=1&tags=Food%2CReview&title=A+review+of+my+favorite+Italian+restaurant&content=I+got+the+sausage+orchette+and+it+was+amazing.+I+wish+that+instead+of+baby+broccoli+they+used+rapini.+Otherwise+it+was+a+perfect+dish+and+the+vibes+were+great And got a new post with the id 11. UPDATE and DELETE operations are pretty similar to what we’ve seen so far. Most complexity in your queries will be similar to what we’ve seen in the posts query where we want to JOIN or GROUP BY data in various ways. To update the post we can write a query that looks like this: ` COALESCE acts similarly to if we had written a ?? b in JavaScript. If the binded value that we provide is null it will fall back to the default. We can delete our new post with a simple DELETE query: ` Transactions / Batching One thing to note with D1 is that I don’t think the traditional style of SQLite transactions are supported. You can use the db.batch API to achieve similar functionality though. According to the docs: Batched statements are SQL transactions ↗. If a statement in the sequence fails, then an error is returned for that specific statement, and it aborts or rolls back the entire sequence. ` Summary In this post, we've taken a hands-on approach to exploring the D1 Client API, starting with defining our database schema and loading seed data. We then dove into writing queries, covering the basics of prepared statements and parameter binding, before moving on to more complex topics like joins and transactions. We saw how to construct and execute queries to fetch data from our database, including how to handle relationships between tables and marshal result data into a usable format. We also touched on inserting, updating, and deleting data, and how to use transactions to ensure data consistency. By working through these examples, we've gained a solid understanding of how to use the D1 Client API to interact with our database and build robust, data-driven applications....

Dec 23, 2024

6 mins

JavaScript

Let's innovate together!

We're ready to be your trusted technical partners in your digital innovation journey.

Whether it's modernization or custom software solutions, our team of experts can guide you through best practices and how to build scalable, performant software that lasts.

Git Strategies for Working on Teams

Background

Always Branch and Pull Request to Main

Merges to main via Squash and Merge

Individual Branching

Long Running Epic Branches

Conclusion

Dustin Goodman

You might also like

"How do I undo my most recent commit?" - Mastering the git reset command

Mastering Git Rerere: Solving Repetitive Merge Conflicts with Ease

The Dangers of ORMs and How to Avoid Them

D1 SQLite: Writing queries with the D1 Client API

Let's innovate together!

You might also like

"How do I undo my most recent commit?" - Mastering the git reset command

Mastering Git Rerere: Solving Repetitive Merge Conflicts with Ease

The Dangers of ORMs and How to Avoid Them

D1 SQLite: Writing queries with the D1 Client API